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Highlights 


Highlights  of  GAO-1 1  -676,  a  report  to 
congressional  committees 


DEFENSE  MANAGEMENT 

Actions  Needed  to  Improve  Management  of  Air 
Force’s  Food  Transformation  Initiative 


Why  GAO  Did  This  Study 


What  GAO  Found 


To  assess  quality  of  life  and  identify 
areas  for  improvement,  the  Air  Force 
conducts  periodic  surveys  of  its 
airmen.  Recent  surveys  indicted  that 
many  of  them  were  not  satisfied  with 
the  quality,  variety,  and  availability  of 
food  at  Air  Force  bases.  As  a  result, 
the  Air  Force  implemented  an  initiative 
in  2010 — the  Food  Transformation 
Initiative — to  improve  on-base  food 
services.  The  Air  Force  piloted  the 
initiative  at  six  installations  with  a 
$10.3  million  contract  awarded  in 
August  2010  and  plans  to  eventually 
expand  the  initiative  to  other 
installations  in  the  United  States.  The 
Senate  and  House  Armed  Services 
Committees  directed  GAO  to 
undertake  a  comprehensive  review  of 
the  initiative.  GAO  reviewed  (1)  the 
initiative’s  objectives  and  performance 
measures,  (2)  implementation 
progress,  (3)  the  initiative’s  impact  on 
food  service  workers  at  the  pilot 
installations,  and  (4)  alternative 
approaches  the  Air  Force  considered. 
GAO  reviewed  documentation  on  the 
initiative,  interviewed  Air  Force  and 
contractor  officials,  and  visited  four  of 
the  six  pilot  installations. 


What  GAO  Recommends 

GAO  recommends  that  the  Air  Force 
take  several  actions  to  improve 
management  of  the  Food 
Transformation  Initiative,  including 
developing  an  evaluation  plan  to 
assess  the  initiative’s  results  before 
moving  beyond  the  pilot.  In 
commenting  on  a  draft  of  this  report, 
the  Department  of  Defense  concurred 
with  all  the  recommendations. 


The  Air  Force  established  eight  objectives  for  its  Food  Transformation  Initiative, 
but  it  has  not  fully  developed  metrics  or  an  evaluation  plan  for  assessing  the 
initiative’s  results.  When  GAO  began  its  review,  the  Air  Force  had  developed 
metrics  for  three  of  the  eight  objectives.  The  Air  Force  subsequently  identified 
metrics  for  four  additional  objectives,  but  it  did  not  have  a  robust  evaluation  plan, 
as  called  for  by  best  practices  derived  from  prior  GAO  work  and  various  guidance 
documents,  that  describes  how  the  initiative’s  results  will  be  measured.  Without 
well-defined  metrics,  contained  in  an  evaluation  plan  that  is  clearly  linked  to  its 
objectives,  the  Air  Force  is  not  in  a  position  to  adequately  assess  the  results  of 
the  pilot  and  make  informed  decisions  about  the  initiative’s  future. 

The  Air  Force  has  made  some  improvements  to  its  food  service  operations  as  a 
result  of  the  Food  Transformation  Initiative,  but  a  key  part  of  the  initiative  has 
been  delayed,  and  it  is  too  early  to  measure  its  results.  The  Air  Force  has 
obtained  mostly  positive  feedback  on  the  changes  it  has  made,  but  the  data 
collected  to  date  are  preliminary.  In  addition,  the  campus  dining  concept,  which 
will  allow  airmen  to  use  their  meal  cards  to  eat  at  on-base  dining  facilities  other 
than  the  main  dining  facility,  had  not  been  implemented  at  the  time  of  GAO’s 
review.  Therefore,  this  concept,  a  key  part  of  the  initiative,  could  not  be 
assessed.  Furthermore,  initial  data  show  that  the  Air  Force  may  have 
underestimated  some  of  the  costs  of  the  initiative,  by  overestimating  the  number 
of  hours  military  personnel  can  provide  the  contractor  for  cooking  and  food 
preparation. 

The  Food  Transformation  Initiative,  as  implemented  so  far,  has  had  varying 
effects  on  food  service  workers.  All  military  and  civilian  cooks  at  the  main  dining 
facilities  have  maintained  their  jobs,  but  some  expressed  concerns  about 
increases  in  their  job  responsibilities.  Most  contract  mess  attendant  employees 
have  retained  their  jobs,  but  the  total  number  of  these  employees  has  been 
reduced  and  some  contract  employees  lost  their  jobs  as  a  result  of  the  initiative. 
Further,  most  civilian  employees  at  nonappropriated  fund  food  and  beverage 
operations,  such  as  clubs  and  snack  bars,  accepted  jobs  with  the  new  contractor, 
but  their  job  security  remains  uncertain  as  the  contractor  tries  to  make  the 
facilities  profitable  and  the  remaining  parts  of  the  initiative  are  implemented. 

The  Air  Force  included  three  options  in  its  analysis  of  alternatives  in  selecting  its 
method  for  improving  its  food  services,  but  did  not  follow  its  own  guidance  for 
identifying  and  assessing  alternatives  that  would  meet  the  program’s  stated 
objectives.  GAO  found  that  two  of  the  three  alternatives  were  not  viable  options 
to  be  considered  because  they  only  met  three  of  the  program’s  eight  objectives. 

In  addition,  the  Air  Force  may  be  able  to  reduce  its  food  service  costs  by 
reviewing  and  renegotiating  its  existing  food  service  contracts  at  bases  that  are 
not  part  of  the  pilot  program. 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


July  26,  2011 

Congressional  Committees 

Maintaining  satisfaction  with  quality  of  life  is  a  key  ingredient  in  the  Air 
Force’s  ability  to  successfully  retain  needed  personnel.  To  assess  quality 
of  life  and  identify  areas  where  improvements  are  needed,  the  Air  Force 
conducts  periodic  surveys  of  airmen.  Through  these  surveys,  the  Air 
Force  found  many  airmen  were  not  satisfied  with  the  quality,  variety,  and 
availability  of  food  at  Air  Force  installations.  In  an  effort  to  improve 
satisfaction  in  this  area,  the  Air  Force  developed  a  program — the  Food 
Transformation  Initiative — designed  to  improve  the  quality,  variety,  and 
availability  of  food  while  continuing  the  training  of  military  cooks  who  feed 
airmen  during  deployments.  In  October  2010,  the  Air  Force  began  piloting 
the  initiative  at  six  Air  Force  installations  and  plans  to  eventually  expand  it 
to  other  installations  in  the  United  States.1 

On  August  31 , 201 0,  the  Air  Force  awarded  the  contract  for  the  Food 
Transformation  Initiative  to  Aramark,  a  large  company  experienced  in 
food  service,  and  on  October  1,  2010,  issued  a  delivery  order  for  an 
estimated  $10.3  million.2  The  contract  covers  a  full  range  of  dining 
services,  including  cleaning  the  facility  and  menu  development,  as  well  as 
providing  input  into  the  design  of  renovations  for  some  of  the  existing 
dining  facilities  at  the  six  pilot  bases.  It  is  anticipated  that  the  renovations 
to  the  main  dining  facilities,3  expected  to  cost  an  additional  $9  million,  will 
generally  be  paid  for  by  the  Air  Force,  subject  to  funding  caps.  While  the 
initiative  is  focused  on  improving  food  and  service  at  the  main  dining 
facilities,  a  second  element  is  to  have  the  contractor  operate  the 
nonappropriated  fund  food  and  beverage  operations,  such  as  clubs  and 
bowling  center  snack  bars,  because  many  of  these  facilities  were  losing 


’The  pilot  installations  include  Elmendorf  Air  Force  Base,  Alaska;  Little  Rock  Air  Force 
Base,  Arkansas;  Fairchild  Air  Force  Base,  Washington;  Travis  Air  Force  Base,  California; 
MacDill  Air  Force  Base,  Florida;  and  Patrick  Air  Force  Base,  Florida. 

2ln  a  decision  issued  August  2,  2010,  we  denied  a  bid  protest  concerning  the  Food 
Transformation  Initiative  solicitation.  See  John  P.  Santry — Designated  Employee  Agent, 
B-402827,  Aug.  2,  2010,  2010  CPDfl  177. 

3The  Air  Force  calls  its  main  dining  facilities  “mission  essential  feeding  facilities,”  which  is 
also  the  term  used  in  the  contract.  We  use  the  term  main  dining  facilities  to  refer  to  these 
facilities  in  this  report. 
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money  and  in  danger  of  being  closed.  By  including  the  nonappropriated 
fund  food  and  beverage  operations  in  the  initiative,  the  Air  Force  will 
eventually  be  able  to  allow  the  installations  to  institute  “campus  style 
dining,”  where  airmen  can  use  their  meal  cards  to  purchase  food  at 
different  facilities  at  each  installation.4 

In  reports  accompanying  proposed  bills  for  the  National  Defense 
Authorization  Act  for  fiscal  year  201 1 ,5  the  Senate  and  House  Armed 
Services  Committees  directed  that  we  undertake  a  comprehensive  review 
of  the  Air  Force’s  Food  Transformation  Initiative.  In  response,  we 
reviewed  (1)  the  initiative’s  objectives  and  performance  measures, 

(2)  progress  the  Air  Force  has  made  in  meeting  the  objectives  of  the 
initiative,  (3)  the  initiative’s  impact  on  food  service  employees  at  the  pilot 
bases,  and  (4)  the  Air  Force’s  consideration  of  alternative  approaches  for 
transforming  its  food  service  program. 

To  address  these  objectives,  we  interviewed  Air  Force  officials  about  the 
development  of  objectives  and  performance  measures  for  the  initiative 
and  assessed  the  objectives  and  performance  measures  in  view  of 
Department  of  Defense  (DOD)  and  Air  Force  guidance  and  criteria  from 
our  previous  work.  In  addition,  we  collected  and  analyzed  data  related  to 
the  initiative’s  objectives,  such  as  changes  in  the  number  of  meals  served 
and  the  level  of  airmen’s  satisfaction  with  food  services.  To  assess  the 
reliability  of  these  data,  we  discussed  the  procedures  for  generating  and 
verifying  the  data  with  knowledgeable  officials  and,  where  possible, 
examined  the  data  for  obvious  anomalies.  Where  we  found  limitations 
with  the  data,  we  presented  that  information  in  the  report,  but  otherwise 
determined  that  the  data  were  sufficiently  reliable  for  the  purposes  of  our 
report.  We  also  interviewed  contractor  officials,  union  representatives, 
and  food  service  employees  about  their  experiences  under  the  initiative 
and  reviewed  information  on  the  impact  the  initiative  has  had  on  civilian 
food  service  employees,  military  food  service  personnel,  food  service 
contract  workers,  and  employees  at  nonappropriated  fund  food  and 
beverage  operations.  Further,  we  assessed  the  process  the  Air  Force 


4Certain  junior  enlisted  personnel,  primarily  in  grades  E1-E4  and  who  are  assigned  to  the 
dormitory,  are  usually  placed  on  essential  station  messing.  Enlisted  personnel  on 
essential  station  messing  receive  meals  in  the  dining  facility  by  presenting  proper 
identification.  In  this  report,  we  use  the  term  meal  card  to  refer  to  these  benefits. 

5 See  S.  Rep.  No.  111-201,  at  115  (2010)  (accompanying  S.  3454);  H.R.  Rep.  No.  111-491 
at  264-265  (2010)  (accompanying  H.R.  5136). 
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used  in  selecting  the  Food  Transformation  Initiative,  including  alternatives 
that  were  considered.  We  compared  the  Air  Force’s  business  case 
analysis,  which  describes  the  alternatives  considered,  to  the  criteria 
provided  in  the  GAO  Cost  Estimating  and  Assessment  Guide* * * * *  6  and  Air 
Force  guidance.  We  also  discussed  with  Air  Force  officials  options  other 
than  the  Food  Transformation  Initiative  they  could  have  considered  for 
improving  food  service  at  other  installations.  We  conducted  this 
performance  audit  from  September  2010  to  July  201 1 ,  in  accordance  with 
generally  accepted  government  auditing  standards.  Those  standards 
require  that  we  plan  and  perform  the  audit  to  obtain  sufficient,  appropriate 
evidence  to  provide  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives.  We  believe  that  the  evidence  obtained 
provides  a  reasonable  basis  for  our  findings  and  conclusions  based  on 
our  audit  objectives.  A  more  detailed  description  of  our  scope  and 
methodology  is  included  in  appendix  I. 


Background 


In  the  Air  Force,  food  service  program  oversight  is  provided  by  the  Air 
Force  Services  Agency,  headquartered  in  San  Antonio,  Texas.  The 

agency  is  responsible  for  oversight  of  the  Air  Force’s  food  service 
program,  including  developing  the  menus  for  the  main  dining  facilities  and 
providing  training  for  military  food  service  workers  who  assist  in  food 
preparation  and  the  operation  of  dining  facilities.  According  to  Air  Force 

officials,  each  Air  Force  installation  contracts  for  mess  attendant  services, 

such  as  cleaning  and  cashiering  services,  at  their  main  dining  facilities 
and  is  responsible  for  awarding  and  managing  these  contracts.  In 
addition,  although  many  installations  use  both  military  and  civilian 
personnel  as  cooks,  according  to  Air  Force  officials,  some  installations 
have  expanded  their  mess  attendant  services  contracts  to  enable  them  to 
use  contract  employees  to  prepare  and  serve  food.  Depending  on  the 
availability  of  and  priorities  given  to  contractors  located  near  the 
installations,  according  to  Air  Force  officials,  the  installations  may  award 

these  food  service  contracts  to  small  business  firms,7  contractors  covered 


6GAO,  GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
Managing  Capital  Program  Costs,  GAO-09-3SP  (Washington,  D.C.:  March  2009). 

According  to  the  Small  Business  Administration,  a  food  service  contractor  is  considered  a 
small  business  if  its  average  annual  receipts  are  equal  to  or  less  than  $35.5  million. 
Information  on  calculating  annual  receipts  is  provided  in  13  C.F.R.  §  121.104(c).  Prior  to 
November  5,  2010,  the  threshold  for  food  service  contractors  was  $20.5  million.  See  Small 
Business  Size  Standards;  Accommodation  and  Food  Service  Industries,  75  Fed.  Reg. 
61,604  (Oct.  6,  2010)  (to  be  codified  at  13  C.F.R.  pt.  121). 
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by  the  Randolph-Sheppard  Act,8  or  contractors  covered  by  the  Ability  One 
program.9 


The  Air  Force  Services  Agency,  which  developed  the  Food 
Transformation  Initiative  and  awarded  the  implementation  contract  for  the 
six  pilot  installations  to  Aramark,  is  responsible  for  managing  and 
overseeing  the  execution  of  the  contract,  along  with  a  contracting  officer’s 
representative  from  each  base’s  Force  Support  Squadron.  Under  the 
contract,  Aramark  is  responsible  for  overseeing  the  day-to-day  food 
service  operations  at  the  six  pilot  installations  and  planning  and  executing 
the  menus.  Aramark  is  also  responsible  for  subcontracting  for  the 
required  food  service  labor  at  the  pilot  installations  and  is  required  under 
its  contract  to  subcontract  with  Ability  One  agencies  for  the  main  dining 
facilities.10  Aramark  works  with  the  Ability  One  agencies  to  negotiate  the 
labor  hours  needed  and  the  cost  of  labor  at  the  pilot  installations. 

Air  Force  officials  plan  to  implement  the  initiative  at  a  second  set  of  up  to 
eight  installations  after  assessing  the  results  of  the  initiative  at  the  six 
installations  in  the  pilot  program.  If  the  initiative  continues  to  be 
successful,  they  plan  to  implement  it  at  more  Air  Force  installations  in  the 
United  States  over  the  next  5  years.  Potentially,  the  Food  Transformation 
Initiative  could  be  implemented  at  all  of  the  Air  Force’s  149  appropriated 
fund  dining  facilities  nationwide. 


sThe  Randolph-Sheppard  Act  created  a  vending  facility  program  in  1936  to  provide  blind 
individuals  with  more  job  opportunities  and  to  encourage  their  self-support.  See  Pub.  L. 
No.  74-732  (1936)  (codified  as  amended  at  20  U.S.C.  §§  107  to  107f).  The  program  trains 
and  employs  blind  individuals  to  operate  vending  facilities  on  federal  property.  Federal 
law  gives  blind  vendors  under  the  program  a  priority  to  operate  cafeterias  on  federal 
property.  See  20  U.S.C.  §  107d-3(e).  According  to  Air  Force  officials,  the  Air  Force 
currently  has  nine  Randolph-Sheppard  Act  contractors  operating  dining  facilities. 

9The  Ability  One  program,  formerly  referred  to  as  the  Javits-Wagner-O’Day  Act  program 
and  codified  at  41  U.S.C.  §§  8501  to  8506,  provides  training  and  employment 
opportunities  for  persons  who  are  blind  or  have  other  severe  disabilities  through  the 
purchase  of  products  and  services  from  nonprofit  agencies  employing  these  individuals.  If 
a  product  or  service  is  on  the  procurement  list  maintained  by  the  Committee  for  Purchase 
from  People  Who  Are  Blind  or  Severely  Disabled,  a  government  entity  intending  to 
procure  that  item  or  service  generally  must  procure  it  from  a  qualified  nonprofit  agency. 
See  §  8504.  Federal  Acquisition  Regulation  provisions  implement  the  program.  See  48 
C.F.R.  subpart  8.7. 

10The  contract  contains  a  Statement  of  Objectives  for  each  pilot  location,  each  of  which 
indicates  that  Ability  One  participation  is  not  required  at  nonappropriated  fund  food  and 
beverage  resale  operation  locations  but  is  encouraged. 
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Initially,  the  Food  Transformation  Initiative  focused  on  improving  food 
service  at  the  pilot  installations’  main  dining  facilities  by  increasing  the 
hours  of  service,  remodeling  the  dining  facilities  (using  money  provided 
by  the  Air  Force),  developing  new  menus,  and  providing  an  executive 
chef  at  each  location.  In  addition,  the  main  dining  facilities  were  opened  to 
all  installation  personnel.  Previously,  the  main  dining  facilities  at  most 
installations  were  not  available  to  all  installation  personnel.  According  to 
Air  Force  officials,  the  potential  increase  in  customers  by  opening  the 
dining  facilities  to  all  installation  personnel  is  a  key  component  of  the 
initiative  because  making  greater  use  of  a  facility,  which  has  fixed 
overhead  costs,  could  help  defray  the  costs  of  providing  meals. 

According  to  the  Air  Force,  the  initiative  also  offered  installation 
commanders  the  option  of  turning  over  their  nonappropriated  fund  food 
and  beverage  operations,  such  as  bowling  center  and  golf  course  snack 
bars  and  officers’  and  enlisted  members’  clubs,  to  Aramark  in  a 
commission-based  contract.11  Table  1  lists  the  pilot  installations  and 
dining  facilities  included  in  the  Food  Transformation  Initiative  contract. 


Table  1:  Dining  Facilities  at  Each  Pilot  Installation 


Air  Force  Nonappropriated  fund  food  and 


installation 

Main  dining  facilities 

beverage  operations3 

Elmendorf 

•  Iditarod  Dining  Facility 

•  Kenai  Flight  Kitchen 

Eagle’s  Nest  Cafe  at  the  Eagle 
Glen  Golf  Course 

The  Warehouse  Grill  at  the 

Kashim  Enlisted  Club 

Paradise  Cafe  at  the  Community 
Center 

Hillberg  Snack  Barb 

Fairchild 

•  Warrior  Inn 

•  Warrior  Inn  Express 

•  Ross  Dining  Facility 

•  Flightline  Feeding  Facility 

Funspot  Bowling  Center  Snack 

Bar 

Comingled  lounge  at  the  Bowling 
Center0 

Final  Point  at  the  Club  Annex 

Little  Rock 

•  Hercules  Dining  Facility 

•  Flight  Line  Kitchen 

Deer  Run  Golf  Course 

Strike  Zone  Cafe  at  the  Bowling 
Center 

Hangar  1080  Club 

"The  contract  and  orders  under  the  contract  use  the  terms  concessionaire  or  commission 
rate  in  describing  the  nature  of  the  contract. 
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Air  Force 
installation 


Nonappropriated  fund  food  and 
beverage  operations3 


MacDill 


Patrick 


T  ravis 


Main  dining  facilities 

•  Diner’s  Reef 

•  Flight  Kitchen  (colocated 
with  Diners’  Reef) 


•  Riverside  Dining  Facility 

•  Riverside  Flight  Kitchen 


•  Sierra  Inn 

•  Sierra  Inn  Grab  and  Go 

•  Golden  Bear  Flight 
Kitchen 


Bay  Palms  Golf  Course  Snack 
Bar 

MacDill  Lanes  Snack  Bar  at  the 
Bowling  Center 

Bayshore  Officers’  Club 

Boomer’s  Bar  and  Grill  -  Enlisted 
Club 

Seascapes  Beach  Club  -  All 
Ranks  Club 

Manatee  Cove  Golf  Course  Cafe 
and  Lounge 

Rocket  Lanes  Ten  Pin  Cafe 
The  Tides  Club  and  The 
Blockhouse  -  Enlisted  Club 

Gatsby’s  Grill  at  the  Golf  Course 

Strike  Zone  Grill  at  the  Bowling 
Center 

Delta  Breeze  Colocated  Club 
Rickenbacker’s  Coffee  Shop 


Source:  GAO  analysis  of  Air  Force  information. 

aAs  represented  by  the  Air  Force. 

bThe  Statement  of  Objectives  includes  seasonal  operation  of  the  Hillberg  T-Bar  Grill  among  the 
potential  future  dining  opportunities,  available  for  incorporation  into  the  delivery  order  at  the 
installation’s  discretion.  A  modification  to  the  initial  nonappropriated  fund  order  for  Elmendorf  adds  a 
line  item  to  allow  the  contractor  to  operate  the  Hillberg  facility  snack  bar  from  February  7  to  March  31 , 
2011,  based  on  specified  terms  and  conditions.  The  modification  states  that  “[n]o  commissions  will  be 
paid  due  to  the  partial  season  and  timeline”  and  that  a  “reassessment  of  the  operation  will  occur  after 
completion  of  the  season  before  additional  seasons  are  addressed.” 

cThis  facility  does  not  appear  on  the  nonappropriated  fund  order  with  the  contractor,  but  the 
Statement  of  Objectives  for  Fairchild  Air  Force  Base  indicates  that  the  installation  desires  to  develop 
a  comingled  bar/lounge  adjacent  to  the  snack  bar. 


Traditionally,  nonappropriated  fund  food  and  beverage  operations  have 
been  part  of  the  installations’  morale,  welfare,  and  recreation  program  run 
by  nonappropriated  fund  personnel  employed  by  nonappropriated  fund 
instrumentalities  at  each  Air  Force  installation.12  According  to  Air  Force 
officials,  these  food  and  beverage  operations  have  been  losing  money, 
and  including  them  in  the  contract  with  Aramark  was  an  attempt  to  make 


12Nonappropriated  fund  instrumentalities  are  organizations  that  typically  provide  for  the 
morale,  welfare,  and  recreation  of  government  officers  and  employees.  Employees  of 
armed  forces  nonappropriated  fund  instrumentalities  are  generally  not  deemed  to  be 
employees  of  the  United  States  except  as  specifically  provided  by  statute. 
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them  more  profitable.  For  example,  according  to  Air  Force  data,  the 
nonappropriated  fund  food  and  beverage  operations  at  the  six  pilot 
installations  lost  almost  $950,000  during  the  5-year  period  from  fiscal  year 
2005  to  fiscal  year  2009.  In  fiscal  year  2009,  the  clubs,  golf  course  snack 
bars,  and  bowling  center  snack  bars  at  the  Travis  and  Elmendorf  Air 
Force  bases  lost  about  $122,000  and  $99,000  respectively.  In  the  new 
arrangement,  Aramark  operates  the  facilities,  including  hiring  personnel  to 
run  the  operations  and  developing  menus,  and  plans  to  invest  its  own 
money  to  remodel  some  of  these  nonappropriated  fund  food  and 
beverage  operations.  In  return,  the  nonappropriated  fund  instrumentality 
will  receive  a  commission  on  each  sale.  At  each  of  the  pilot  locations,  the 
installation  commanders  decided  to  turn  over  operation  of  many  of  their 
nonappropriated  fund  food  and  beverage  operations  to  Aramark. 


The  Air  Force  Has 
Established 
Objectives  for  the 
Food  Transformation 
Initiative,  but  Has  Not 
Fully  Developed 
Metrics  or  an 
Evaluation  Plan  for 
Assessing  Results 


The  Air  Force  has  established  eight  objectives  for  its  Food 
Transformation  Initiative,  but  it  has  not  fully  developed  the  metrics  or  an 
evaluation  plan  for  assessing  the  initiative’s  results.  Best  practices,  such 
as  those  prompted  by  principles  derived  from  our  previous  work,13  and 
from  DOD  and  Air  Force  guidance,14  suggest  that  programs  need  specific 
metrics  against  which  their  performance  can  be  measured.  These  metrics 
should  be  well  defined  and  include  measurable  targets.  Our  previous 
work  further  suggests  that,  ideally,  these  objectives  and  metrics  should  be 
described  in  a  single  document,  such  as  an  evaluation  plan  that  defines 
how  results  can  be  measured.  This  is  especially  important  for  a  pilot 
program  so  that  performance  information  can  be  used  to  make  changes 
and  adjustments  to  make  the  program  more  effective  before  expanding  its 
implementation  to  other  installations.  Without  an  evaluation  plan  with  well- 


13See  GAO,  Agency  Performance  Plans:  Examples  of  Practices  That  Can  Improve 
Usefulness  to  Decisionmakers,  GAO/GGD/AIMD-99-69  (Washington,  D.C.:  Feb.  26, 
1999).  Although  this  report  focuses  on  agency  plans  under  the  Government  Performance 
and  Results  Act,  it  presents  useful  principles  for  measuring  program  performance. 

14For  example,  although  DOD  Instruction  5000.02,  which  includes  guidance  on  the 
acquisition  of  services,  may  not  directly  apply  to  the  Food  Transformation  Initiative,  it 
provides  useful  information  concerning  considering  performance  evaluation  in  the  context 
of  acquisition  planning.  See  DODI  5000.02,  Operation  of  the  Defense  Acquisition  System, 
enc.  9,  §  4(f)  (Dec.  8,  2008).  Similarly,  although  officials  indicated  that  Air  Force  guidance 
regarding  performance-based  acquisition  of  services  would  not  apply  in  the  context  of  the 
Food  Transformation  Initiative,  the  guidance  provides  principles  that  we  believe  would  be 
useful  in  the  context  of  the  initiative’s  objectives.  See,  for  example,  Air  Force  Instruction 
63-124,  Performance-Based  Services  Acquisition  (Aug.  1, 2005). 
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defined  metrics  that  are  clearly  linked  to  the  objectives,  the  Air  Force  may 
not  be  in  a  position  to  adequately  assess  the  results  of  the  pilot  program 
and  make  informed  decisions  about  the  initiative’s  future. 


The  Air  Force  Developed 
Eight  Objectives  for  Its 
Food  Transformation 
Initiative 


The  Air  Force  developed  eight  objectives  for  its  Food  Transformation 
Initiative,  as  shown  in  table  2. 15  Some  of  the  objectives  focus  on  activities 
that  expand  food  options  for  airmen  and  other  personnel  with  access  to 
the  installation,  while  others  focus  on  training,  efficiencies,  and  costs.  For 
example,  part  of  one  of  the  objectives  is  to  open  the  main  dining  facilities 
to  everyone  on  the  installation  and  to  allow  airmen  to  use  their  meal  cards 
at  locations  other  than  the  main  dining  facility. 


Table  2:  Food  Transformation  Initiative  Objectives 


Objective 

Description 

1. 

Enhance  food  quality, 
variety,  and  availability 

Improve  the  visual  appearance  and  taste  of  menu  items; 
offer  healthy  food;  increase  menu  items;  increase  hours. 

2. 

Maintain  home  base 
and  war  fighting 
feeding  capabilities 

Ensure  that  airmen  receive  experience  and  training  in  food 
service  and  are  ready  to  deploy. 

3. 

Focus  on  airmen’s 
changing  needs, 
lifestyles  and 
preferences 

Better  align  food  service  with  the  needs  of  the  customers. 

4. 

Enhance  sense  of 
community 

Make  the  main  dining  facility  a  community  food 
establishment,  making  airmen  feel  more  a  part  of  their 
community  by  allowing  them  to  use  their  meal  card 
privileges  at  food  outlets  other  than  the  main  dining  facility. 

5. 

Improve  programs  and 
facilities 

Renovate  dining  facilities  to  the  atmosphere  that  is  similar 
to  what  is  found  on  college  campuses;  bring  additional  food 
service  availability  to  the  installation  by  enhancing  remote 
feeding  capabilities. 

6. 

Provide  enhanced 
nutritious  meals 

Continue  to  provide  nutritious  food  that  meets  DOD 
nutritional  standards. 

7. 

Improve  efficiency 

Improve  food  service  delivery  approaches  to  improve 
efficiency. 

8. 

Decrease  costs 

Maintain  or  decrease  the  cost  of  operating  the  main  dining 

facilities. 


Source:  GAO  analysis  of  Air  Force  information. 


15These  objectives  are  not  explicitly  provided  in  the  contract,  but  rather  are  contained  in 
Air  Force  briefing  materials  and  other  documents. 
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The  Air  Force  Has  Not 
Fully  Developed  Metrics 
and  an  Evaluation  Plan  to 
Use  in  Assessing  the 
Initiative’s  Results 


Our  initial  review  of  the  Air  Force’s  documentation  on  the  Food 
Transformation  Initiative  showed  it  had  limited  information  on  how  it 
planned  to  measure  implementation  of  the  objectives  and,  ultimately,  the 
results  of  the  pilot  program.  We  initially  found  that  the  Air  Force  had 
defined  metrics  for  three  of  the  initiative’s  eight  objectives:16 

•  To  measure  progress  on  its  first  objective — enhancing  food  quality, 
variety,  and  availability — the  Air  Force  plans  to  use  its  annual 
customer  satisfaction  survey  and  set  a  goal  score  of  70  out  of  100  for 
the  survey  questions  pertaining  to  hours  of  operation,  food  variety, 
food  pricing,  and  the  overall  dining  experience.17  However,  the  Air 
Force’s  annual  customer  satisfaction  survey  could  be  biased  and 
therefore  may  not  provide  the  data  it  needs  to  measure  results  for  this 
objective  because  the  response  rate  for  recent  surveys  has  been  low. 
For  example,  although  58,805  active  duty  airmen  responded  to  the 
survey,  the  response  rate  was  about  17  percent.  Further,  the  Air 
Force  did  not  perform  a  nonresponse  analysis  to  determine  whether  it 
is  likely  that  those  who  did  not  respond  to  the  survey  would  have 
provided  substantively  different  answers  from  those  who  responded. 

•  To  measure  progress  in  meeting  its  seventh  objective — improving 
efficiency — the  Air  Force  plans  to  track  and  monitor  Aramark’s 
quarterly  financial  statements  to  identify  whether  the  contractor  was 
adhering  to  its  original  budget  for  labor  costs.  The  Air  Force  defined 
this  objective  as  improving  food  service  delivery  to  improve  efficiency 
in  maintaining  food,  labor,  and  nonfood  costs.  However,  based  on  the 
documentation  we  reviewed,  it  was  not  clear  that  a  specific  goal  had 
been  established  for  the  various  cost  elements. 

•  For  its  eighth  objective — decreasing  costs — the  Air  Force  plans  to  use 
information  from  the  same  sources  to  measure  results  as  the 
information  for  its  seventh  objective,  improving  efficiency.  Specifically, 
the  Air  Force  plans  to  monitor  Aramark’s  quarterly  financial 
statements  to  track  the  operating  expenses  of  the  dining  facilities.  The 
Air  Force  currently  defines  the  metric  for  this  objective  as  maintaining 
or  decreasing  costs  as  compared  to  previous  food  service  contracts. 


16As  with  the  Food  Transformation  Initiative  objectives,  these  metrics  were  not  clearly 
contained  in  the  initial  contract,  but  rather  provided  to  us  in  written  responses  to  our 
questions  to  Air  Force  officials. 

17Survey  respondents  were  asked  to  rank  items  on  a  scale  of  1  to  10,  with  1  considered 
poor  and  10  considered  excellent.  These  rankings  were  then  converted  to  scores  of  0  to 
100. 
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However,  initially  the  Air  Force  estimated  that  the  Food 
Transformation  Initiative  would  achieve  a  30  percent  cost  savings 
when  compared  to  existing  food  service  contracts.  Since  that  time,  the 
Air  Force  has  reduced  that  estimate  to  27  percent.  However,  neither 
figure  was  being  used  as  an  actual  metric  for  measuring  results  for 
this  objective. 


During  the  course  of  our  review,  we  brought  to  the  Air  Force’s  attention 
the  lack  of  defined  metrics  for  all  objectives  and  an  evaluation  plan  for 
measuring  progress  in  achieving  the  initiative’s  objectives.  As  a  result,  the 
Air  Force  developed  four  additional  metrics  for  its  objectives. 

•  To  measure  progress  in  meeting  its  second  objective — maintaining 
home  base  and  warfighting  feeding  capabilities — the  Air  Force  plans 
to  monitor  the  readiness  of  military  food  service  personnel  to  deploy 
using  DOD’s  readiness  system.  The  goal  is  to  achieve  a  readiness 
score  of  C-1 ,  which  means  that  these  personnel  have  the  required 
resources  and  are  trained  to  undertake  the  full  mission  for  which  they 
are  trained  without  any  compensation  for  deficiencies. 

•  To  measure  progress  in  meeting  its  third  objective — to  focus  on 
airmen’s  changing  needs,  lifestyles,  and  preferences — the  Air  Force 
plans  to  use  its  annual  customer  satisfaction  survey.  Specifically,  the 
Air  Force  has  defined  goal  scores  for  two  specific  survey  sections: 
food  and  satisfaction.  Under  the  food  section  of  the  survey,  the  Air 
Force  has  defined  a  goal  score  of  76  out  of  100  for  the  part  titled  “type 
of  food  you  want.”  Under  the  satisfaction  section  of  the  survey,  the  Air 
Force  will  use  the  score  on  how  well  the  dining  facility  rates 
“compared  to  expectations”  and  has  defined  a  goal  score  of  76.  In 
addition,  the  Air  Force  will  measure  the  number  of  available  meals 
that  meal  card  holders  eat  on  the  installation,  with  a  goal  of  having 
meal  card  holders  eat  70  percent  of  available  meals. 

•  For  its  fifth  objective — to  improve  programs  and  facilities — the  Air 
Force  will  again  use  its  annual  customer  satisfaction  survey  to 
measure  results  and  has  a  goal  score  of  85  out  of  100  on  the  facilities 
section  of  the  survey. 

•  To  measure  progress  in  meeting  its  sixth  objective — providing 
enhanced  nutritious  meals — the  Air  Force  has  defined  a  target  score 
of  80  out  of  100  on  the  “availability  of  healthy  food  choices”  question 
in  the  customer  satisfaction  survey. 


At  the  time  we  completed  our  review,  the  Air  Force  had  not  yet  defined  a 
metric  for  its  fourth  objective — to  enhance  a  sense  of  community.  The  Air 
Force  stated  that  it  will  use  its  annual  customer  satisfaction  survey  to 
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measure  the  effect  of  increased  use  of  the  dining  facility  on  having  a 
sense  of  community,  but  it  had  not  defined  goal  scores  in  this  area. 

Most  of  the  evidence  we  obtained  on  the  Air  Force’s  planned 
assessments  of  the  initiative  was  in  the  form  of  written  responses  to  our 
inquiries  and  a  modification  to  a  delivery  order  under  the  contract  that  laid 
out  some  specific  goals  the  Air  Force  had  developed  for  the  contractor. 
The  modification  contains  an  incentive  fee  plan  with  key  performance 
indicators  that  will  be  used  to  determine  the  availability  of  an  incentive 
fee.  Several  of  the  key  performance  indicators  may  link  to  performance 
metrics  for  the  objectives,18  but  neither  the  contract  itself  nor  the  delivery 
order  explicitly  outlines  the  eight  Food  Transformation  Initiative  objectives 
or  provides  all  performance  metrics  to  measure  those  objectives.  At  the 
time  of  our  review,  Air  Force  officials  stated  that  they  did  not  have 
planning  documents  that  lay  out  the  metrics,  data  sources,  or  planned 
assessment  schedules  for  the  pilot  program.  They  also  told  us  that  they 
recognized  the  need  to  further  refine  their  metrics  and  develop  an 
evaluation  plan. 


The  Food 
Transformation 
Initiative  Has 
Produced  Some  Food 
Service 

Improvements,  but 
More  Time  Is  Needed 
to  Measure  Results 


Although  the  Air  Force  has  made  some  improvements  to  its  food  service 
operations  as  a  result  of  the  Food  Transformation  Initiative,  it  is  too  early 
to  measure  the  results  of  the  initiative.  The  Air  Force  has  obtained  mostly 
positive  feedback  on  the  changes  it  has  implemented,  but  the  data 
collected  to  date  are  preliminary,  and  parts  of  the  initiative  have  not  been 
implemented.  For  example,  implementation  of  a  key  part  of  the  initiative, 
the  campus  dining  concept,  which  will  allow  airmen  to  use  their  meal 
cards  to  eat  at  food  service  locations  other  than  the  main  dining  facility, 
has  been  delayed  from  January  201 1  and  had  not  been  implemented  at 
the  time  of  our  review.  Furthermore,  the  data  we  reviewed  on  labor  costs 
for  the  first  6  months  of  the  Aramark  contract  indicated  that  the  actual 
savings  from  the  initiative  may  be  less  than  the  Air  Force’s  initial  30 
percent  cost  savings  estimate.  In  its  initial  estimates  of  the  costs  of  the 
initiative,  it  appears  that  the  Air  Force  overestimated  the  number  of  hours 


1sFor  example,  one  of  the  key  performance  indicators  would  require  a  review  of  customer 
satisfaction  scores  in  the  categories  of  hours  of  operation,  variety,  pricing,  and  overall 
dining  experience.  The  overall  rating  must  exceed  a  customer  satisfaction  index  of  75  and 
any  one  survey  factor  must  not  fall  below  a  score  of  70.  Another  key  performance 
indicator  would  measure  performance  against  the  annual  budget  in  reducing  contractor 
nonfood  costs  per  meal. 
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military  personnel  would  be  able  to  work  in  the  dining  facilities  and 
underestimated  the  wages  of  contract  employees.  As  a  result,  it  is  unclear 
whether  the  Air  Force  will  successfully  meet  its  objective  of  reducing 
costs  as  a  result  of  the  initiative. 


The  Air  Force  Has  Made 
Improvements  to  Its  Food 
Service  Operations  and 
Progress  in  Meeting  Some 
Objectives  of  the  Food 
Transformation  Initiative 


The  Air  Force  has  made  improvements  to  its  food  service  operations  and 
cited  progress  in  meeting  five  of  its  eight  objectives  for  the  Food 
Transformation  Initiative  at  the  six  pilot  installations.  Specifically,  officials 
stated  that  they  have  made  significant  improvements  to  the  menus  and 
increased  the  hours  of  operation,  which  has  led  to  increased  customer 
satisfaction.  According  to  Air  Force  officials,  these  activities  show 
progress  in  meeting  three  of  the  program’s  objectives:  enhancing  food 
quality,  variety,  and  availability  (objective  1);  providing  enhanced 
nutritious  meals  (objective  6);  and  focusing  on  airmen’s  changing  needs, 
lifestyles,  and  preferences  (objective  2). 


To  achieve  its  goals,  Aramark  added  a  pizza  station,  improved  the  salad 
bar,  and  enhanced  the  deli  menu  in  the  main  dining  halls.  In  addition, 
Aramark  revitalized  the  menus,  adding  new  recipes,  such  as  chicken 
marsala  and  shrimp  scampi,  while  keeping  some  traditional  Air  Force 
recipes  that  were  popular  among  the  airmen.  According  to  the  Air  Force 
officials,  more  healthy  cooking  options  are  now  available  because  the 
menu  selections  include  more  fresh  vegetables.  At  each  of  the  four  bases 
we  visited,  Air  Force  officials  stated  that  there  has  been  increased 
customer  satisfaction  with  the  quality  of  the  food. 


Air  Force  officials  also  stated  that  increasing  the  hours  of  operation  to 
make  food  service  more  available  was  an  integral  part  of  the  objective  to 
enhance  food  quality,  variety,  and  availability.  Under  the  Food 
Transformation  Initiative,  the  six  pilot  installations  have  increased  the 
hours  of  operation  at  their  main  dining  facilities.  The  hours  of  operation 
increased  to  1 1 2  hours  per  week  at  the  main  dining  facilities  at  each  base 
except  Patrick  Air  Force  Base,  which  does  not  have  a  requirement  for  a 
midnight  meal  period.19  Officials  at  several  of  the  installations  we  visited 
noted  that  the  customers  were  pleased  with  the  expanded  hours. 


19The  midnight  meal  is  designed  to  provide  food  to  airmen  who  work  overnight  shifts  and 
cannot  eat  at  normal  designated  meal  times. 
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Table  3:  Increase  in  Hours  of  Operation  at  the  Main  Dining  Facilities 


Air  Force  base 

Number  of  weekly  hours 
before  the  Food 
Transformation  Initiative 

Number  of  weekly  hours 
after  the  Food 
Transformation  Initiative 

Increase  in  number 
of  weekly  hours 

Percentage  increase 
in  weekly  hours 

Elmendorf 

66.0 

112.0 

46.0 

69.7% 

Fairchild 

58.5 

112.0 

53.5 

91.5% 

Little  Rock 

45.5 

112.0 

66.5 

146.2% 

MacDill 

56.3 

112.0 

55.8 

99.1% 

Patrick 

42.0 

98.0 

56.0 

133.3% 

T  ravis 

63.0 

112.0 

49.0 

77.8% 

Source:  GAO  analysis  of  Air  Force  data. 


The  data  the  Air  Force  plans  to  use  to  measure  its  progress  in  meeting 
these  three  objectives  are  from  its  annual  customer  satisfaction  survey. 
According  to  the  most  recent  customer  satisfaction  survey,  conducted 
from  December  2010  to  January  2011,  survey  respondents  at  the  six  pilot 
installations  indicated  higher  levels  of  satisfaction  than  respondents  at  Air 
Force  bases  that  had  not  implemented  the  Food  Transformation  Initiative 
in  the  areas  of  having  the  type  of  food  they  wanted,  hours  of  operation, 
and  availability  of  healthy  food  choices  (see  fig.  1).  The  respondents 
indicated  higher  levels  of  satisfaction  at  the  six  pilot  locations,  but  the 
survey  goals  of  76  out  of  100  for  “type  of  food  you  want”  and  80  out  of 
100  for  “availability  of  healthy  food  choices”  were  not  met.  The  goal  of  70 
out  of  100  for  “hours  of  operation”  was  achieved.  Air  Force  officials  note 
they  used  this  survey,  as  well  as  additional  information  gathered  from  the 
private  sector,  to  help  define  goal  scores  and  will  reevaluate  their  goals 
each  year  of  the  program.  However,  as  noted  previously,  due  to  the  low 
response  rate,  these  survey  data  may  not  be  reliable  enough  for 
comparison  purposes. 
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Figure  1:  Comparison  of  Customer  Satisfaction  Survey  Results  for  Pilot  Bases  to 
All  Air  Force  Bases,  2010 


Score 

100 

90 

80  78  78 


Type  of  food  Hours  of  Availability  of 

you  want  operation  healthy  food  choices 

Survey  questions 


Food  Transformation  Initiative  locations 
Non-Food  Transformation  Initiative  locations 
Source:  GAO  analysis  of  Air  Force  data. 


Note:  Survey  respondents  were  asked  to  rank  items  on  a  scale  of  1  to  10,  with  1  considered  poor  and 
10  considered  excellent.  These  rankings  were  then  converted  to  scores  of  0  to  100.  Due  to  the  low 
response  rate,  these  survey  data  may  not  be  reliable  enough  for  comparison  purposes. 


Air  Force  officials  stated  that,  while  there  is  still  work  to  be  done,  they 
have  started  to  transform  the  dining  facilities  to  create  an  atmosphere 
similar  to  what  is  found  on  college  and  university  campuses,  which  is  an 
important  part  of  objective  5 — improve  programs  and  facilities.  Although 
major  renovations  of  the  main  dining  facilities  have  not  begun,  Aramark 
has  made  some  changes,  such  as  enhancing  food  displays  and 
improving  the  appearance  of  dining  facility  entryways.  The  photographs  in 
figures  2  and  3  illustrate  improvements  made  to  the  entryway  and  the 
salad  bar  at  MacDill  Air  Force  Base. 
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Figure  2:  Entryway  at  Dining  Facility,  MacDill  Air  Force  Base,  Florida,  before  and  after  Initiative 


Source:  Air  Force. 
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Figure  3:  Salad  Bar  at  MacDill  Air  Force  Base,  Florida,  before  and  after  Implementation  of  the  Food  Transformation  Initiative 


Source:  Air  Force. 

The  Air  Force  plans  to  spend  almost  $9  million  in  renovations  to  the  main 
dining  facilities  at  the  six  pilot  installations.  Air  Force  officials  stated  that 
they  expect  the  renovations  to  begin  in  June  or  July  201 1 .  Plans  for 
renovating  the  dining  facilities  include  expanding  the  cashier  area  to  allow 
for  more  cashiers  during  peak  periods,  removing  walls  to  open  up  the 
food  service  area  and  create  a  more  efficient  customer  flow,  adding  more 
seating  capacity,  and  making  general  cosmetic  upgrades.  Table  4  lists 
the  projected  costs  of  renovations  at  each  of  the  six  pilot  installations. 


Table  4:  Projected  Cost  of  Dining  Facility  Renovations  at  Six  Pilot  Installations 

Air  Force  base 

Cost  (dollars) 

Elmendorf 

$1,238,439 

Fairchild 

2,176,729 

Little  Rock 

865,029 

MacDill 

1,610,745 

Patrick 

1,402,358 

T  ravis 

1,696,545 

Total 

$8,989,845 

Source:  Air  Force. 
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Another  objective  in  which  Air  Force  officials  said  they  have  made  some 
progress  is  objective  4 — enhancing  a  sense  of  community.  They  told  us 
an  essential  part  of  this  objective  is  to  open  the  dining  facilities  to 
everyone  on  the  base.  Prior  to  the  initiative,  five  of  the  six  pilot 
installations  allowed  only  active  duty  enlisted  personnel  to  dine  at  the 
main  dining  facilities.  Installation  officials  noted  that  opening  the  main 
dining  facilities  up  to  everyone  who  has  access  to  the  installation  has  had 
a  very  positive  effect,  improving  the  sense  of  community. 

Table  5  shows  a  comparison  of  the  number  of  meals  purchased  at  the 
pilot  installations  for  the  first  6  months  after  implementing  the  initiative 
(October  2010  to  March  2011)  to  the  same  6-month  period  the  previous 
year.  Over  250,000  more  meals  were  purchased  in  the  main  dining 
facilities  after  the  initiative  was  implemented.  Over  187,000  of  those 
meals  reflected  an  increase  in  the  number  of  cash  paying  customers. 


Table  5:  Change  in  the  Number  and  Percentage  of  Meals  Purchased  at  the  Main  Dining  Facilities 


Increase  (decrease)  in  total  meals  purchased 


Total  meals  purchased,  Total  meals  purchased, 

October  2009  through  October  2010  through  Meals  purchased  by  Meals  purchased  by 

March  2010  March  2011  meal  card  customers  cash  customers 


Base 

Meals 
purchased  by 
meal  card 
customers 

Meals 
purchased 
by  cash 
customers 

Meals 
purchased  by 
meal  card 
customers 

Meals 
purchased 
by  cash 
customers 

Number 

Percentage 

Number 

Percentage 

Elmendorf 

94,979 

38,511 

98,941 

64,715 

3,962 

4 

26,204 

68 

Fairchild 

85,716 

54,998 

86,549 

76,602 

833 

1 

21,604 

39 

Little 

Rock 

68,914 

38,546 

111,991 

46,853 

43,077 

63 

8,307 

22 

MacDill 

38,073 

58,494 

42,814 

111,576 

4,741 

12 

53,082 

91 

Patrick 

23,012 

33,776 

17,614 

95,612 

(5,398) 

(23) 

61,836 

183 

T  ravis 

132,659 

46,455 

150,106 

62,981 

17,447 

13 

16,526 

36 

Total 

443,353 

270,780 

508,015 

458,339 

64,662 

15 

187,559 

69 

Source:  GAO  analysis  of  Air  Force  data. 


Meal  card  holders  have  always  been  able  to  use  the  main  dining  facilities, 
but  Air  Force  officials  said  they  believe  the  increase  in  usage  is  a  further 
indication  of  the  effect  of  the  increased  hours  of  availability  and  improved 
food  choices.  The  number  of  airmen  choosing  to  use  their  meal  card 
benefits  at  the  main  dining  facility  after  implementation  of  the  initiative 
increased  at  five  of  the  installations,  including  Little  Rock  Air  Force  Base, 
where  customers  using  meal  cards  increased  by  more  than  60  percent. 
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Air  Force  officials  attributed  part  of  the  increase  to  an  increase  in  the 
number  of  airmen  at  the  installation  during  this  period,  but  also  to  the 
expanded  menus  and  increased  hours  that  resulted  from  the  initiative. 
Only  Patrick  Air  Force  Base  had  a  decrease  in  the  number  of  meal  card 
customers  served.  Air  Force  officials  attributed  this  change  to  a  decrease 
in  the  number  of  airmen  at  the  installation  during  this  period.  The  Air 
Force  had  not  fully  developed  a  metric  for  measuring  progress  in  meeting 
this  objective  and  had  not  implemented  all  changes  related  to  this 
objective,  but  Air  Force  officials  stated  that,  based  on  the  increased 
usage  of  the  dining  facilities,  they  believe  some  progress  has  been  made. 


Additional  Time  Is  Needed 
to  Measure  the  Results  of 
the  Food  Transformation 
Initiative 


Although  the  Air  Force  reported  improvements  to  its  food  service 
program,  it  is  too  early  to  measure  the  results  of  the  Food  Transformation 
Initiative,  even  at  the  pilot  installations.  According  to  service  officials,  data 
on  the  impact  of  changes  made  to  date  are  preliminary  and  may  not 
predict  future  results.  For  example,  Air  Force  and  base  officials  stated 
that  some  of  the  increases  in  the  number  of  customers  may  be  due  to  a 
“honeymoon  phase”  in  which  customers  visit  the  main  dining  facility 
because  it  is  new  or  different.  Thus,  there  may  be  some  decrease  in  the 
number  of  customers  as  the  newness  of  the  changes  in  the  dining 
facilities  wear  off.  Installation  officials  also  told  us  that  increased 
satisfaction  with  the  food  selection  and  hours  of  operation  may  also 
decrease  over  time.  Customers  have  expressed  some  dissatisfaction 
already,  primarily  with  the  increase  in  prices  for  cash-paying  customers. 
As  part  of  the  Food  Transformation  Initiative,  the  Air  Force  raised  the 
prices  paid  by  cash  customers  at  the  main  dining  facilities  by  increasing 
the  surcharge  paid  on  the  cost  of  food  to  90  percent.20  Previously,  cash 
customers  paid  the  cost  of  food  plus  a  60  percent  surcharge.  According  to 
Air  Force  officials,  this  price  increase  helps  offset  the  cost  of  the  contract. 
Officials  at  the  four  installations  we  visited  noted  that  cash-paying 
customers  expressed  some  dissatisfaction  with  the  price  increases. 
According  to  the  Air  Force’s  recent  customer  satisfaction  survey,  the  six 
pilot  installations  scored  lower  on  satisfaction  with  the  price  of  food  at  the 
main  dining  facilities  compared  to  other  Air  Force  installations.  Air  Force 
officials  said  that  they  have  adjusted  some  prices  in  response  to  the 


20The  contract  statement  of  work  lists  two  categories  of  patrons  for  the  main  dining 
facilities:  essential  station  messing  customers,  to  be  fully  reimbursed  by  the  government 
as  part  of  the  delivery  order  and  all  other  eligible  patrons,  considered  “cash  customers.” 
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A  Key  Part  of  the 
Initiative — the 
Campus  Dining 
Concept — Has  Not 
Been  Implemented 


negative  comments,  especially  for  the  salad  bar,  and  will  continue  to 
make  necessary  adjustments  as  they  obtain  more  customer  feedback. 


At  the  time  of  our  review,  the  campus  dining  concept,  in  which  airmen  will 
be  able  to  use  their  meal  cards  to  eat  at  dining  facilities  on  the  base  other 
than  the  main  dining  facility,  had  not  been  implemented  at  any  of  the  six 
pilot  bases.  The  Air  Force  originally  expected  this  part  of  the  initiative  to 
be  implemented  in  January  201 1 .  This  was  later  changed  to  April  201 1 
and  then  to  June  and  July  201 1 .  Air  Force  officials  told  us  they  expected 
to  implement  campus  dining  at  Patrick  Air  Force  Base  in  June  201 1  and 
at  the  remaining  five  pilot  bases  in  July  201 1 . 

Implementation  of  the  campus  dining  concept  was  delayed  for  several 
reasons,  according  to  Air  Force  officials,  including  the  need  to  develop  a 
policy  on  how  the  campus  dining  concept  will  work — such  as  what 
purchases  will  be  allowed  for  those  using  meal  cards  at  facilities  other 
than  the  main  dining  facilities — and  delays  in  developing  the  software 
needed  to  track  all  meal  card  purchases.  Air  Force  officials  told  us  they 
only  recently  acquired  the  software  needed  to  track  meal  card  purchases 
and  were  beginning  to  test  it.  In  addition,  they  upgraded  the  hardware,  if 
necessary,  for  the  main  dining  facilities  and  bought  additional  hardware  to 
support  campus  dining  at  some  of  the  nonappropriated  fund  food  and 
beverage  operations.  When  operational,  this  software  will  allow  the  Air 
Force  to  determine  how  often  and  where  airmen  use  their  meal  cards.  In 
addition,  Air  Force  officials  told  us  that  they  recently  completed 
negotiations  with  Aramark  on  the  menu  options  that  will  be  available  for 
airmen  to  purchase  with  their  meal  cards  at  the  nonappropriated  fund 
food  and  beverage  operations. 

Officials  at  Elmendorf  Air  Force  Base  raised  additional  concerns  about 
the  campus  dining  concept  because  the  installation  is  a  joint  base  with 
the  Army’s  Fort  Richardson.  At  the  time  of  our  review,  soldiers  stationed 
at  Fort  Richardson  were  allowed  to  use  their  meal  card  privileges  at  the 
main  dining  facility  at  Elmendorf  Air  Force  Base.  Installation  officials  said 
they  would  like  that  privilege  to  be  extended  to  the  nonappropriated  fund 
food  and  beverage  operations  under  the  campus  dining  concept  and  did 
not  want  to  implement  campus  dining  until  this  issue  had  been  resolved. 
Air  Force  officials  told  us  that,  initially,  campus  dining  was  going  to  be 
restricted  to  Air  Force  personnel  stationed  at  Elmendorf  Air  Force  Base. 

The  campus  dining  concept  is  an  important  part  of  the  Food 
Transformation  Initiative  and  of  meeting  the  Air  Force’s  fourth  objective — 
to  enhance  the  sense  of  community — by  allowing  airmen  to  use  more  of 
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the  base  community  when  eating  their  daily  meals.  The  Air  Force  recently 
addressed  the  issues  that  led  to  the  delay  of  the  campus  dining  concept, 
but  it  had  not  fully  implemented  this  part  of  the  initiative  at  the  time  of  our 
review.  Therefore,  the  Air  Force  cannot  assess  the  concept  or  make 
necessary  adjustments  to  this  key  part  of  the  initiative. 


Initial  Labor  Cost 
Information  Indicates 
Savings  May  Be  Less 
Than  Estimated 


It  is  too  early  to  determine  cost  savings  resulting  from  the  Food 
Transformation  Initiative,  but  initial  information  on  labor  costs  indicates 
that  actual  savings  may  be  less  than  the  30  percent  cost  savings 
originally  estimated  by  the  Air  Force.  The  Air  Force  underestimated 
contract  labor  costs  because  it  overestimated  the  number  of  military  labor 
hours  it  could  provide  to  the  contractor  for  cooks  and  food  preparers. 
Specifically,  the  Air  Force  estimated  that  military  personnel  would  be  able 
to  provide  approximately  32,000  hours  per  month  of  labor  in  the  dining 
facilities  at  the  six  pilot  installations.  However,  the  military  personnel  for 
whom  these  estimates  were  provided  have  not  been  able  to  provide  the 
projected  number  of  hours  to  the  contractor  because  of  other  duties.  As  a 
result,  Aramark  has  hired  additional  contract  staff  to  make  up  some  of  the 
difference,  which  could  increase  contract  labor  costs. 


We  found  that,  for  the  first  6  months  of  the  contract,  the  number  of  military 
labor  hours  was  about  14,000  hours  less  than  originally  estimated  for  the 
six  pilot  installations.  At  Elmendorf  Air  Force  Base,  labor  costs  were 
projected  to  increase  by  $105,000  for  the  remainder  of  the  fiscal  year  as  a 
result  of  fewer  military  personnel  being  available.  Air  Force  officials  told 
us  they  plan  to  hold  Aramark  to  its  original  budget,  but  they  indicated  that, 
in  certain  situations,  after  Aramark  provides  adequate  documentation,  the 
Air  Force  will  pay  the  higher  labor  costs. 

The  Air  Force  also  underestimated  the  subcontractor  labor  costs  to  be 
incurred  by  Aramark.  For  example,  we  found  that,  for  the  first  quarter  of 
fiscal  year  201 1 ,  Aramark  submitted  requests  for  reimbursement  for 
subcontractor  labor  costs  that  were  about  $400,000  more  than  those 
originally  estimated  by  the  Air  Force.  For  the  second  quarter  of  fiscal  year 
2011,  subcontractor  labor  costs  submitted  for  reimbursement  were  about 
$600,000  more  than  originally  estimated.  According  to  the  Air  Force, 
these  costs  may  be  the  result  of  a  pay  raise  resulting  from  an  annual 
wage  determination  that  was  unaccounted  for  in  the  original  budget  and 
not  under  the  control  of  Aramark.  Air  Force  officials  told  us  that  they  have 
requested  supporting  documentation  for  the  higher  labor  costs  but  have 
not  paid  this  amount  to  Aramark.  However,  the  officials  indicated  that, 
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with  adequate  support  from  the  subcontractor,  the  Air  Force  would  pay 
the  higher  labor  costs. 


Implementation  of  the 
Food  Transformation 
Initiative  Has  Had 
Varying  Effects  on 
Food  Service  Workers 


Preliminary  results  indicate  that  the  Food  Transformation  Initiative  has 
had  varying  effects  on  the  three  categories  of  food  service  workers  at  the 
six  pilot  bases:  (1)  appropriated  fund  civilian  and  military  cooks  in  the 
main  dining  halls,  (2)  contract  mess  attendant  workers  in  the  main  dining 
halls,  and  (3)  nonappropriated  fund  civilian  employees  who  work  in  the 
nonappropriated  fund  food  and  beverage  operations.  All  civilian 
employees  and  military  cooks  who  worked  in  the  main  dining  halls 
retained  their  jobs,21  but  some  employees  expressed  concern  to  us  and 
their  union  representatives  that  their  job  responsibilities  under  the 
initiative  have  increased.  Although  most  contract  mess  attendants 
retained  their  jobs,  the  total  number  of  contract  mess  attendant 
employees  was  reduced  and  some  employees  lost  their  jobs  as  a  result 
of  the  Food  Transformation  Initiative.  Finally,  most  civilian  employees  at 
the  nonappropriated  fund  food  and  beverage  operations  accepted 
employment  with  Aramark,  but  their  job  security  remains  uncertain 
because  if  these  operations  do  not  become  more  profitable,  they  may 
close. 


Civilian  and  Military  Cooks 
Retained  Their  Jobs,  but 
Their  Responsibilities 
Increased 


All  appropriated  fund  civilian  employees  and  military  food  service  workers 
at  the  main  dining  halls  retained  their  jobs,  but  some  employees 
expressed  concern  that  they  had  taken  on  increased  responsibilities  as  a 
result  of  the  initiative.  At  the  six  pilot  bases,  202  military  and  20  civilian 
employees  worked  in  the  main  dining  facilities  prior  to  the  implementation 
of  the  Food  Transformation  Initiative.  These  workers  were  not  affected  by 
the  Food  Transformation  Initiative  contract;  they  continue  to  be  employed 
directly  by  the  Air  Force.  According  to  Air  Force  officials,  the  workers’ 
primary  responsibilities  are  cooking  and  serving  food. 


During  our  visits  to  four  of  the  pilot  installations,  we  spoke  with  small 
groups  of  civilian  and  military  food  service  workers  and  individual  workers 


21During  the  course  of  the  bid  protest  on  the  Food  Transformation  Initiative,  the  Air  Force 
amended  the  solicitation  to  provide  that  appropriated  fund  wage  grade  employees  would 
not  be  “displaced,  reassigned,  subjected  to  reduction  in  force,  or  otherwise  adversely 
affected  by  the  implementation  of  FTI  phase  I.”  See  John  P.  Santry — Designated 
Employee  Agent,  B-402827,  Aug.  2,  2010,  2010  CPD  IT  177  at  5  (citing  Letter  to  Offerors, 
June  30,  2010,  HI. a). 
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who  were  available  at  the  time  of  our  visit  and  agreed  to  speak  with  us.  In 
addition,  we  interviewed  civilian  union  representatives  about 
implementation  of  the  initiative  at  the  four  installations  we  visited.  They 
raised  some  concerns  about  the  increased  workloads  of  these  workers 
and  noted  that,  prior  to  the  Food  Transformation  Initiative,  contract 
workers  performed  some  of  the  food  preparation  work,  such  as  chopping 
vegetables  and  replenishing  the  salad  bar.  Since  implementation  of  the 
initiative,  this  work  has  now  become  part  of  the  civilian  and  military  cooks’ 
job  responsibilities.  In  addition,  because  the  new  menus  rely  on  an 
increased  amount  of  fresh  ingredients,  there  has  been  an  increase  in  the 
amount  of  food  preparation  they  must  perform.  While  these  tasks  were 
new  to  the  civilian  and  military  cooks,  according  to  Air  Force  officials,  they 
should  have  been  performing  these  tasks  prior  to  implementation  of  the 
initiative.  Contract  workers  were  less  affected  by  these  changes  because, 
according  to  Air  Force  officials,  over  time,  they  had  taken  on  additional 
duties  that  might  not  have  been  in  their  original  job  descriptions.  Air  Force 
officials  also  told  us  the  initiative  helped  clarify  the  required 
responsibilities  of  the  civilian  and  military  cooks  and  those  of  the  contract 
employees. 

In  addition,  some  military  food  service  workers  at  the  installations  we 
visited  told  us  that,  because  of  the  extended  hours  and  increase  in 
number  of  meals  served,  there  is  no  down  time  and  they  are  no  longer 
getting  to  take  their  lunch  time  or  other  breaks.  They  also  said  they  are 
now  expected  to  do  physical  training  and  other  required  training  on  their 
own  time  rather  than  during  work  hours  as  was  the  case  prior  to  the 
initiative.  Some  of  the  military  cooks  to  whom  we  spoke  said  they  are 
putting  in  12-  or  13-hour  days  to  keep  up  with  the  increased  workload. 
They  also  told  us  they  are  feeding  many  more  people  than  prior  to 
implementation  of  the  initiative,  but  they  have  the  same  number  of  people 
cooking.  Aramark  officials  stated  that  they  are  continuing  to  determine  the 
appropriate  size  of  dining  facility  operations,  including  the  number  of 
required  labor  hours,  and  will  make  adjustments  as  more  information  is 
gathered. 
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Most  Contract  Mess 
Attendant  Workers  Were 
Retained  under  the 
Initiative,  but  Some 
Workers  Lost  Their  Jobs 


Most  contract  mess  attendant  workers  in  the  main  dining  facilities  at  the 
six  pilot  installations  retained  their  jobs  after  implementation  of  the  Food 
Transformation  Initiative.  However,  some  contract  mess  attendant 
workers — Ability  One  workers22  and  small  business  employees — lost  their 
jobs  as  a  result  of  the  initiative,  and  others’  work  hours  were  reduced. 
Prior  to  the  initiative,  five  of  the  six  pilot  installations  used  Ability  One 
providers  for  their  mess  attendant  contracts  and  the  other  installation 
contracted  with  a  small  business  to  provide  mess  attendants  for  its  dining 
halls.  According  to  Air  Force  officials,  after  implementation  of  the  initiative, 
Aramark  hired  some  Ability  One  workers  to  provide  mess  attendant 
services  at  the  sixth  installation  in  place  of  the  small  business  employees. 


Following  implementation  of  the  initiative,  the  five  installations  continued 
to  use  Ability  One  workers  as  mess  attendants  through  subcontracts 
between  Aramark  and  Ability  One  providers.  However,  although  many  of 
the  Ability  One  workers  retained  their  jobs,  the  total  number  of  workers 
was  reduced  at  four  of  the  five  installations.  According  to  Air  Force 
officials,  the  five  pilot  installations  that  initially  employed  Ability  One 
workers  had  273  of  these  workers  prior  to  the  initiative.  As  of  March  201 1 , 
the  number  of  Ability  One  workers  at  these  five  installations  decreased  to 
233  (see  table  6).  According  to  Air  Force  officials,  72  Ability  One  workers 
initially  lost  their  jobs  as  a  result  of  the  Food  Transformation  Initiative. 
However,  they  noted  that,  after  the  initial  layoffs,  32  new  Ability  One 
workers  were  hired  at  the  five  bases  as  Aramark  adjusted  staffing  levels 
at  the  main  dining  facilities.  The  officials  could  not  tell  us,  however, 
whether  any  of  the  32  new  Ability  One  workers  hired  were  among  the  72 
workers  who  initially  lost  their  positions  after  implementation  of  the 
initiative. 


Table  6:  Number  of  Ability  One  Workers  before  and  after  the  Food  Transformation  Initiative 

Air  Force  base 

Number  of  Ability 
One  workers  as  of 
October  1, 2010 

Number  of  Ability  One  workers 
terminated  as  a  result  of  the 
Food  Transformation  Initiative 

Number  of  new  Ability  One 
workers  hired  from  October 
2010  to  March  31,  2011 

Number  of  Ability 
One  workers  as  of 
March  31,  2011 

Elmendorf 

47 

21 

0 

26 

Little  Rock 

63 

26 

1 

38 

22We  use  the  term  Ability  One  workers  to  refer  to  employees  participating  in  the  Ability 
One  program  through  employment  by  a  nonprofit  agency  for  the  blind  or  a  nonprofit 
agency  for  employing  persons  with  severe  disabilities. 
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Air  Force  base 

Number  of  Ability 
One  workers  as  of 
October  1, 2010 

Number  of  Ability  One  workers 
terminated  as  a  result  of  the 
Food  Transformation  Initiative 

Number  of  new  Ability  One 
workers  hired  from  October 
2010  to  March  31,2011 

Number  of  Ability 
One  workers  as  of 
March  31,  2011 

MacDill 

49 

2 

10 

57 

Patrick 

47 

6 

16 

57 

T  ravis 

67 

17 

5 

55 

Total 

273 

72 

32 

233 

Source:  GAO  analysis  of  Air  Force  data. 


In  addition,  a  representative  from  NISH,  one  of  the  central  nonprofit 
agencies  that  facilitates  the  Ability  One  program,  reported  that  the  work 
hours  of  many  of  those  who  retained  their  jobs  were  reduced.  Aramark 
officials  noted  that  the  hours  of  many  contract  employees  were  reduced 
as  the  company  worked  to  achieve  the  staffing  levels  it  deemed 
appropriate  as  it  gained  experience  under  the  new  contract. 

Prior  to  the  initiative,  one  of  the  pilot  installations,  Fairchild  Air  Force 
Base,  contracted  with  a  small  business  to  provide  services  at  its  main 
dining  facility.  According  to  officials,  because  the  Air  Force  requires  that 
installations  under  the  Food  Transformation  Initiative  obtain  mess 
attendant  services  for  their  main  dining  facilities  through  contracts  with 
Ability  One,  as  of  March  1,  2010,  Ability  One  took  over  the  mess 
attendant  services  contract  at  Fairchild  Air  Force  Base.  According  to  Air 
Force  officials  and  Aramark  officials,  prior  to  implementation  of  the  Food 
Transformation  Initiative,  41  workers  employed  by  the  small  business 
contractor  worked  in  the  installation’s  main  dining  facility.  As  a  result  of 
the  initiative,  17  of  these  workers  lost  their  jobs.  According  to  Aramark 
officials,  the  remaining  24  workers  will  be  retained  until  September  30, 

201 1 ,  to  provide  services  such  as  cashiering  and  cooking  and  they  will 
maintain  the  contract  with  the  small  business  contractor  until  that  time. 
Subsequently,  Aramark  may  attempt  to  directly  hire  some  of  these 
workers  as  cashiers  since,  according  to  Air  Force  officials,  the  Air  Force 
does  not  require  Aramark  to  contract  with  Ability  One  for  cashier  services. 
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Most  Former 
Nonappropriated  Fund 
Food  and  Beverage 
Operation  Workers 
Accepted  Jobs  with 
Aramark 


Most  of  the  civilian  employees  at  the  former  nonappropriated  fund  food 
and  beverage  operations  at  the  six  pilot  installations  accepted  jobs  with 
Aramark.  However,  the  future  of  these  facilities  is  uncertain,  and  therefore 
the  job  security  of  these  workers  is  uncertain.  Prior  to  implementation  of 
the  initiative,  a  combination  of  regular  and  flexible  civilian  food  operation 
workers  were  employed  at  the  nonappropriated  fund  food  and  beverage 
operations  at  each  of  the  six  pilot  installations:  1 1 7  regular  and  1 83 
flexible  workers.23  As  shown  in  table  7,  most  of  these  workers  accepted 
jobs  with  Aramark.  According  to  Air  Force  officials,  most  of  these  workers 
now  receive  higher  pay  and  are  eligible  for  increased  benefits  through 
their  employment  with  Aramark.  For  example,  flexible  workers  who  did 
not  receive  any  benefits  while  employed  by  the  nonappropriated  fund 
instrumentality  are  now  eligible  for  benefits.  However,  according  to  Air 
Force  officials,  both  regular  and  flexible  workers  employed  by  Aramark 
may  opt  to  receive  an  additional  $3.50  more  per  hour  in  lieu  of  benefits, 
and  187  workers  accepted  the  higher  pay  rather  than  receiving  benefits. 


Table  7:  Number  of  Former  Nonappropriated  Fund  Food  and  Beverage  Operation 
Employees  Who  Accepted  Positions  with  Aramark 


Air  Force  base 

Type  of  employee 

Number  of  former 
employees 

Number  who 
accepted  position 
with  Aramark 

Elmendorf 

Regular 

13 

9 

Flexible 

14 

6 

Fairchild 

Regular 

2 

1 

Flexible 

17 

11 

Little  Rock 

Regular 

11 

8 

Flexible 

27 

18 

MacDill 

Regular 

26 

23 

Flexible 

47 

31 

23 According  to  Air  Force  Manual  34-310,  Nonappropriated  Fund  Personnel  Management 
and  Administration  Procedures  (Jan.  19,  2011),  regular  employees  are  guaranteed  a 
minimum  of  20  hours  to  a  maximum  of  40  hours  of  work  per  week,  and  they  receive 
benefits.  Flexible  employees  have  work  schedules  that  depend  on  the  needs  of  the 
activity.  These  employees  may  work  a  minimum  of  zero  hours  to  a  maximum  of  40  hours 
per  week,  but  do  not  receive  benefits.  The  version  of  the  Manual  in  existence  prior  to 
January  201 1  contained  the  same  information  regarding  the  two  employment  categories. 
See  Air  Force  Manual  34-310,  Nonappropriated  Fund  Personnel  Program  Management 
and  Administration  Procedures  §  1.7  (Dec.  1,  1995)  (obsolete). 
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Air  Force  base 

Type  of  employee 

Number  of  former 
employees 

Number  who 
accepted  position 
with  Aramark 

Patrick 

Regular 

34 

32 

Flexible 

42 

31 

T  ravis 

Regular 

31 

27 

Flexible 

36 

27 

Total 

Regular 

117 

100 

Flexible 

183 

124 

Source:  Air  Force. 


Regular  workers  who  had  worked  at  the  nonappropriated  fund  food  and 
beverage  operations  for  at  least  1  year  of  continuous  service  were 
generally  eligible  for  severance  pay;  flexible  workers  were  not.  According 
to  Air  Force  officials,  a  regular  worker  could  accept  employment  with 
Aramark  and  also  receive  severance  pay.  According  to  documents 
provided  by  the  Air  Force,  of  the  300  former  nonappropriated  fund  food 
operation  employees,  severance  payments  were  authorized  and  paid  to 
102  workers  at  the  six  pilot  installations  for  a  total  of  over  $750,000. 

In  some  cases,  regular  workers  with  the  nonappropriated  fund  food  and 
beverage  operations  were  earning  credit  towards  federal  nonappropriated 
fund  retirement  benefits.  Under  employment  with  Aramark,  these  workers 
will  not  have  access  to  the  same  retirement  system.  However,  not  all 
former  nonappropriated  fund  food  and  beverage  operation  workers  were 
enrolled  in  the  nonappropriated  fund  retirement  system.  According  to  Air 
Force  officials,  enrollment  in  the  retirement  system  is  not  automatic  and 
many  workers  opt  not  to  enroll  because  of  the  additional  deduction  from 
their  paychecks.  However,  officials  further  stated  that  workers  who  had 
enrolled  in  the  system  and  then  took  jobs  with  Aramark  could  either  defer 
their  retirement  benefits  or  have  their  contributions  returned  to  them, 
depending  on  their  age  and  length  of  service.  Table  8  shows  the  number 
of  workers  at  the  six  pilot  installations’  nonappropriated  fund  food  and 
beverage  operations  who  were  identified  as  eligible  for  and  were  enrolled 
in  the  nonappropriated  fund  retirement  system  prior  to  implementation  of 
the  initiative. 
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Table  8:  Employees  Eligible  for  and  Enrolled  in  the  Nonappropriated  Fund 

Retirement  System 

Air  Force  base 

Number  of  eligible 
employees 

Number  enrolled  in 
retirement  system 

Elmendorf 

13 

5 

Fairchild 

2 

0 

Little  Rock 

11 

1 

MacDill 

26 

23 

Patrick 

35 

24 

T  ravis 

31 

25 

Total 

118 

78 

Source:  GAO  analysis  of  Air  Force  data. 


Note:  Eligibility  was  identified  by  the  Air  Force. 


Air  Force  officials  also  stated  that,  because  many  of  the  nonappropriated 
fund  food  and  beverage  operations  were  losing  money,  the  Air  Force  may 
not  have  been  able  to  keep  some  of  them  open  and  running.  For 
example,  in  2007,  14  employees  from  nonappropriated  fund  food  and 
beverage  operations  at  Elmendorf  Air  Force  Base  were  laid  off  because 
the  operations  were  losing  money.  According  to  Air  Force  officials,  if  the 
Food  Transformation  Initiative  had  not  been  implemented  at  the  six  pilot 
installations,  more  workers  might  have  lost  their  jobs.  Aramark  officials 
stated  that  they  were  making  changes  to  these  operations  and  hope  to 
increase  efficiencies;  however,  they  expressed  concern  about  keeping  all 
of  the  facilities  open  in  the  future.  Employees  with  whom  we  spoke  at  the 
installations  also  expressed  concern  about  the  possibility  of  losing  their 
jobs  in  the  future. 


The  Air  Force’s  Food 
Transformation 
Analysis  of 
Alternatives  Was 
Limited,  and  Other 
Options  for  Achieving 
Cost  Savings  May  Be 
Available 


The  analysis  the  Air  Force  conducted  for  selecting  its  Food 
Transformation  Initiative  model  was  very  limited  in  the  options  it 
considered.  The  Air  Force  included  three  options  in  its  business  case 
analysis,  but  did  not  follow  its  own  guidance  for  identifying  and  assessing 
alternatives  that  would  meet  the  program’s  stated  objectives.  In  preparing 
its  business  case  analysis,  the  Air  Force  considered  alternatives  that 
were  not  feasible  ways  of  satisfying  its  objectives  and  did  not  consider 
other  alternatives  that  may  have  been  feasible  options.  The  Air  Force 
officials  who  prepared  the  analysis  stated  that  they  were  unaware  of  the 
guidance  and  that,  due  to  the  limited  time  they  had  to  conduct  the 
analysis,  they  did  not  explore  additional  alternatives  when  it  became 
apparent  that  two  of  the  options  under  consideration  were  not  feasible 
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ways  of  satisfying  the  objectives.  Moreover,  the  Air  Force  may  have 
opportunities  for  reducing  costs  at  all  of  its  installations  using  methods  it 
did  not  consider  as  part  of  its  business  case  analysis  for  the  pilot 
program. 


Some  Alternatives  the  Air 
Force  Considered  Were 
Not  Viable,  and  It  Did  Not 
Consider  All  Alternatives 


Air  Force  guidance  on  business  case  analysis  procedures  calls  for  the 
inclusion  of  feasible  alternatives  to  satisfy  the  objectives  and  suggests 
that  all  feasible  alternatives  should  be  considered.  Where  an  alternative 
was  considered  but  dismissed  as  infeasible,  the  guidance  directs  a 
discussion  of  the  reasons  for  considering  it  infeasible.24  Further,  best 
practices  for  preparing  business  case  analyses,  such  as  those  described 
in  the  GAO  Cost  Estimating  and  Assessment  Guide,25  state  that  all 
meaningful  alternatives  should  be  presented  and  analyzed  in  a  consistent 
manner.  However,  the  Air  Force  considered,  and  included  in  its  business 
case  analysis,  alternatives  that  were  not  feasible  methods  of  satisfying 
the  objectives  and  did  not  consider  other  alternatives  that  were  feasible. 


The  Air  Force  Services  Agency  prepared  and  presented  a  business  case 
analysis  in  October  2008  with  three  alternatives  to  the  Air  Force  Smart 
Operations  for  the  21st  Century26  office,  which  provided  start-up  funding 
for  the  Food  Transformation  Initiative.  However,  two  of  the  alternatives 
considered  were  not  viable  options  because  they  did  not  satisfy  all  eight 
objectives  of  the  program. 


The  first  alternative  presented  was  to  close  four  dining  facilities  at 
selected  Air  Force  installations  and  put  all  eligible  airmen  stationed  at 
these  installations  on  basic  allowance  for  subsistence.27  According  to  the 
business  case  analysis,  this  option  would  potentially  save  money  by 
reducing  facility  and  contract  costs,  even  with  the  additional  expense  of 
providing  basic  allowance  for  subsistence  funds  to  airmen.  Our  review  of 
this  option  found  that  it  was  not  viable  because  it  would  meet  only  three  of 


24Air  Force  Manual  65-510,  Business  Case  Analysis  Procedures  (Sept.  22,  2008). 
25GAO-09-3SP. 

26The  Air  Force  Smart  Operations  for  the  21st  Century  program  began  in  2006.  Its  goal 
was  to  integrate  improvement  and  efficiencies  into  Air  Force  programs. 

27Basic  allowance  for  subsistence  is  a  payment  designed  to  defray  the  costs  of  an  eligible 
servicemember’s  meals. 
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the  Air  Force’s  eight  objectives  for  improving  food  service:  maintaining 
home  station  and  warfighting  feeding  capability,  improving  program 
efficiency,  and  reducing  costs.  This  option  would  not  have  satisfied  the 
remaining  five  objectives:  enhancing  food  quality,  focusing  on  airmen’s 
changing  needs,  enhancing  sense  of  community,  improving  programs 
and  facilities,  and  providing  enhanced  nutritious  meals. 

The  second  alternative  presented  in  the  business  case  analysis  was  to 
close  the  four  dining  facilities  presented  in  the  first  option,  centralize  the 
dining  facility  contracts  at  the  other  installations,  and  replace  the  food 
service  contract  workers  with  nonappropriated  fund  employees.  Also,  the 
Air  Force  would  develop  “hybrid”  food  venues  where  airmen  could  use 
their  meal  cards  to  eat  at  locations  other  than  the  main  dining  facilities. 
This  option  would  save  the  same  facility  and  contract  costs  as  the  first 
option  and  further  reduce  contract  costs  by  eliminating  the  food  service 
contracts  at  the  bases  and  having  the  Air  Force  operate  the  dining 
facilities.  The  Air  Force  estimated  that  this  option  would  have  a  43 
percent  return  on  investment.  The  Air  Force  has  operated  some  of  its 
main  dining  facilities  using  nonappropriated  fund  employees  at  other 
bases,  including  Seymour  Johnson  Air  Force  Base  in  North  Carolina. 
However,  similar  to  the  first  alternative,  we  found  that  this  option  also  was 
not  viable  in  that  it  would  meet  only  three  of  the  eight  objectives: 
maintaining  home  station  and  warfighting  feeding  capability,  improving 
program  efficiency,  and  reducing  costs.  Further,  this  option  could  be 
inconsistent  with  a  DOD  policy  providing  that  nonappropriated  fund 
instrumentalities  shall  not  enter  into  contracts  or  agreements  with  DOD 
elements  for  the  provision  of  goods  and  services  that  will  result  in  the  loss 
of  jobs  created  under  Ability  One,  Randolph-Sheppard,  or  small  business 
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programs.28  However,  Air  Force  officials  stated  that  they  may  have  been 
able  to  obtain  a  waiver  of  this  policy  if  this  option  had  been  selected.29 

The  third  alternative  considered  was  to  close  the  four  dining  facilities  and 
develop  hybrid  dining  facilities  to  be  managed  by  a  third-party  food 
service  contractor.  This  option  was  basically  the  same  as  option  two,  but 
with  a  food  service  contractor  taking  over  the  facilities  rather  than  having 
them  being  run  directly  by  personnel  from  the  nonappropriated  fund 
instrumentality.  The  Air  Force  found  that  this  option  would  save  the  most 
money  with  a  108  percent  return  on  investment  and  would  also  meet  all  of 
its  eight  objectives.  Therefore,  this  is  the  option  that  was  chosen. 

In  developing  the  business  case  analysis,  Air  Force  officials  told  us  they 
had  discussed  at  least  one  other  potential  alternative,  but  they  dismissed 
the  idea  as  too  difficult  and  did  not  include  it  in  the  business  case 
analysis.  This  option  was  to  form  a  large  nonappropriated  fund 
instrumentality,  similar  to  the  Army  Air  Force  Exchange  Service,  to  run 
food  service  facilities  at  Air  Force  bases. 

Air  Force  officials  also  told  us  that  they  would  have  included  more 
alternatives  in  the  business  case  analysis,  but  they  did  not  have  enough 
time.  Due  to  the  time  constraints  imposed  by  the  Air  Force  Smart 
Operations  for  the  21st  Century  office  on  the  Air  Force  Services  agency, 
they  had  a  very  limited  time — approximately  2  to  3  weeks — to  complete 
the  analysis.  Because  one  of  the  purposes  of  a  business  case  analysis  is 
to  provide  financial  justification  for  an  investment  decision,  it  is  important 
to  follow  guidance  and  best  practices  in  its  preparation.  However,  even 


28See  Memorandum  from  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness, 
Limitations  on  the  Use  of  Contracts  and  Other  Agreements  with  DOD  Nonappropriated 
Fund  Instrumentalities  (NAFIs)  Pursuant  to  10  U.S.C.  2492  (Dec.  29,  2004).  Section  2492 
of  Title  10  of  the  United  States  Code  provides  authority  for  DOD  nonappropriated  fund 
instrumentalities  to  enter  into  contracts  or  other  agreements  with  other  DOD  elements  in 
certain  circumstances. 

29The  Air  Force  ultimately  requested  and  received  approval  for  a  waiver  to  this  policy  to 
implement  the  Food  Transformation  Initiative  by  using  a  nonappropriated  fund 
instrumentality  memorandum  of  agreement  to  provide  the  installation  appropriated  fund 
food  service  mission.  See  Memorandum  from  the  Under  Secretary  of  Defense  for 
Personnel  and  Readiness,  Request  for  Waiver  to  OSD  Policy  (Aug.  4,  2010).  The  waiver 
approval  recognized  efforts  of  the  Air  Force  to  partner  with  Ability  One,  use  an  acquisition 
strategy  requiring  right  of  first  refusal  for  qualified  small  business  and  nonappropriated 
fund  employees  as  applicable,  and  omit  from  the  initiative  installations  with  existing 
Randolph-Sheppard  Act  dining  facility  contracts.  See  id. 


Page  30 


GAO-1 1-676  Defense  Management 


though  the  Air  Force  has  specific  guidance  on  how  to  prepare  a  business 
case  analysis,  Air  Force  officials  told  us  that  they  were  unaware  of  this 
guidance  as  it  was  not  included  in  the  specific  instructions  from  the  Air 
Force  Smart  Operations  for  the  21st  Century  program. 


The  Air  Force  May  Have 
Other  Options  for  Meeting 
Its  Objective  to  Reduce 
Costs  throughout  Its  Food 
Service  System 


Using  lessons  learned  from  the  implementation  of  the  Food 
Transformation  Initiative,  the  Air  Force  may  have  additional  opportunities 
to  meet  its  objective  to  reduce  costs  throughout  its  food  service  system. 

To  determine  the  impact  of  the  initiative  on  labor  requirements  and  costs, 
we  compared  the  estimated  amount  of  food  service  labor  for  which  the  Air 
Force  contracted  at  the  six  pilot  installations  prior  to  the  initiative  to  the 
amount  for  which  Aramark  planned  to  contract  under  the  initiative.  We 
found  that,  even  with  expanded  hours  of  operation  and  anticipated 
increases  in  the  number  of  meals  served,  Aramark  significantly  reduced 
the  number  of  hours  for  contract  mess  attendant  labor  it  needed  at  the 
pilot  installations. 


According  to  Air  Force  and  Aramark  officials,  most  of  the  cost  savings 
under  the  Food  Transformation  Initiative  resulted  from  reducing  labor 
hours  at  the  main  dining  facilities.30  Prior  to  the  initiative,  the  food  service 
contracts  at  all  six  pilot  installations  were  fixed-price  contracts,  according 
to  Air  Force  officials,  including  for  labor  costs.31  Officials  added  that,  prior 
to  implementation  of  the  initiative,  the  Air  Force  did  not  actually  know  or 
closely  monitor  the  number  of  labor  hours  required  to  provide  food 
services.  However,  we  obtained  documentation  from  the  installations, 
including  memoranda  showing  how  the  prices  were  negotiated  and 
contractor  price  proposals  that  estimated  the  number  of  labor  hours  for 
these  contracts.  Although  these  documents  do  not  contain  the  precise 
number  of  labor  hours  for  the  main  dining  facilities,  they  provided  the  best 
estimates  of  labor  costs  available.  We  compared  the  estimated  labor 
hours  for  these  prior  food  service  contracts  to  those  presented  in  the 
contractor’s  projected  work  schedules  for  the  Food  Transformation 
Initiative  contract  and  found  that  the  total  number  of  labor  hours  at  five  of 


30Air  Force  officials  told  us  that  additional  savings  under  the  Food  Transformation  Initiative 
come  from  optimizing  the  use  of  military  personnel  in  the  main  dining  facilities. 

31Generally,  under  a  firm  fixed-price  contract,  the  installation  pays  the  fixed  price  even  if 
the  actual  total  cost  of  the  product  or  service — that  is,  the  contractor’s  cost  experience  in 
performing  the  contract — falls  short  of  or  exceeds  the  contract  price.  See  48  C.F.R.  § 
16.202-1. 
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the  six  pilot  installations  had  been  reduced  by  53  percent.  For  example,  at 
Travis  Air  Force  Base,  the  number  of  labor  hours  for  the  mess  attendant 
contract  had  been  reduced  by  more  than  half — from  approximately  2,042 
hours  per  week  to  920  hours  per  week.  At  Elmendorf  Air  Force  Base, 
labor  hours  had  been  reduced  from  approximately  1 ,350  hours  per  week 
to  588  hours  per  week.32  Table  9  shows  the  change  in  the  number  of 
labor  hours  at  all  six  pilot  installations. 


Table  9:  Comparison  of  Labor  Hours  under  Previous  Contract  to  Labor  Hours  under 
the  Food  Transformation  Initiative  Contract 

Air  Force 
base 

Estimated  weekly  labor  hours 
under  the  previous  contract 

Estimated  weekly  labor  hours 
under  the  new  contract 

Elmendorf 

1,350 

588 

Fairchild 

979 

476 

Little  Rock 

1,548 

303 

MacDill 

1,201 

1,063 

Patrick 

1,218 

1,349 

T  ravis 

2,042 

920 

Total 

8,338 

4,699 

Source:  GAO  analysis  of  Air  Force  data. 


Patrick  Air  Force  Base  was  the  only  pilot  base  where  the  labor  hours 
were  not  reduced  and  the  only  one  of  the  pilot  installations  where  the 
previous  food  service  contract  had  recently  been  audited.  The  results  of 
that  audit,  conducted  by  the  Air  Force  Audit  Agency  in  2009,  showed  that 
the  food  service  personnel  did  not  align  the  contract  workload  estimates 
with  actual  meals  served.  Specifically,  meal  counts  were  overstated, 
resulting  in  the  installation  paying  more  for  contracted  food  services  than 
necessary.  As  a  result  of  this  audit,  in  October  2009,  Patrick  Air  Force 
Base  renegotiated  its  workload  estimates  and  pay  rates,  resulting  in 
savings  of  approximately  $77,000  annually. 

The  Air  Force  may  have  opportunities  to  reduce  its  food  service  costs  at 
Air  Force  installations  that  are  not  part  of  the  Food  Transformation 
Initiative  pilot  by  reviewing  and,  where  and  when  feasible,  renegotiating 
its  current  food  service  contracts  to  adjust  the  labor  costs.  Air  Force 


32The  labor  hours  presented  here  do  not  include  any  additional  labor  hours  that  might  be 
needed  because  of  a  lack  of  available  military  personnel. 
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officials  told  us  that  they  did  not  previously  consider  this  option  because 
they  did  not  realize  how  poorly  their  food  service  contracts  were 
structured.  Although  renegotiating  the  existing  food  service  contracts 
would  not  meet  all  of  the  objectives  of  the  Food  Transformation  Initiative, 
given  the  large  number  of  food  service  contracts  within  the  Air  Force,  and 
the  length  of  time  it  may  take  to  implement  the  Food  T ransformation 
Initiative  at  all  remaining  Air  Force  installations  in  the  United  States,  the 
Air  Force  may  be  able  to  achieve  significant  savings  prior  to  expanding 
the  initiative.  Throughout  our  site  visits,  at  which  Air  Force  officials 
accompanied  us,  we  discussed  this  potential  opportunity  for  savings.  As  a 
result,  the  Air  Force  recently  issued  a  memorandum  to  the  Major 
Commands  directing  a  review  of  existing  food  service  contracts  to 
determine  if  the  contracts  meet  current  mission  needs.  For  example,  the 
memorandum  indicates  that  special  attention  must  be  given  to  whether 
the  food  service  contract  workload  estimates  were  properly  aligned  with 
the  actual  number  of  meals  served.  We  believe  that  this  is  a  good  first 
step  towards  addressing  this  issue,  but  it  is  important  for  the  Air  Force  to 
continue  to  monitor  this  process  and  continue  to  take  actions  in  this  area, 
where  appropriate. 


Conclusions 


The  Air  Force  is  in  the  process  of  developing  an  approach  for  assessing 
the  results  of  the  Food  Transformation  Initiative,  but  has  not  yet 
developed  well-defined  metrics  that  are  clearly  linked  to  the  objectives  of 
the  initiative  and  adequately  described  in  an  evaluation  plan  to  assess  the 
results  of  the  initiative  at  the  pilot  installations  and  make  informed 
decisions  about  the  initiative’s  future.  Because  the  Air  Force  had  not  yet 
implemented  the  campus  dining  concept  at  the  six  installations  in  the  pilot 
at  the  time  of  our  review,  it  did  not  have  the  information  available  to 
assess  the  results  of  this  key  part  of  the  initiative  to  make  any 
adjustments  that  may  be  needed  before  expanding  the  initiative  to  other 
Air  Force  installations.  In  addition,  until  the  Air  Force  reevaluates  the 
actual  number  of  hours  its  military  personnel  can  provide  the  contractor, 
the  cost  savings  generated  by  the  initiative  will  not  be  clear.  Finally,  by 
reviewing  the  actual  labor  costs  of  its  current  contracts  at  Air  Force 
installations  that  are  not  part  of  the  pilot,  the  Air  Force  may  have 
opportunities  to  reduce  its  food  service  costs  prior  to  expanding  the  Food 
Transformation  Initiative. 
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Recommendations  for  before  implementing  the  Food  Transformation  Initiative  at  other  Air  Force 

installations,  we  recommend  that  the  Secretary  of  the  Air  Force  direct  the 
Executive  Action  Air  Force  Services  Agency  to  take  the  following  three  actions: 

•  Develop  an  evaluation  plan  to  accurately  and  reliably  assess  the 
results  of  the  pilot  phase  of  the  Food  Transformation  Initiative.  This 
evaluation  plan  should  include,  at  a  minimum,  well-defined,  clear,  and 
measurable  metrics  to  use  in  determining  the  results  of  the  initiative 
and  time  frames  for  collecting  data  and  reporting  results. 

•  Evaluate  the  results  of  the  implementation  of  the  campus  dining 
concept  at  the  six  pilot  locations  to  determine  if  adjustments  are 
needed  before  expanding  the  initiative  to  additional  installations. 

•  Reevaluate  the  number  of  military  personnel  hours  the  Air  Force  can 
provide  the  contractor  for  food  service,  make  adjustments  where 
possible  in  projected  labor  hours,  and  take  into  account  the  increased 
labor  costs  in  revising  the  cost  savings  estimates  for  the  future. 


We  further  recommend  that  the  Secretary  of  the  Air  Force  monitor  the 
actions  taken  by  the  Air  Force  Major  Commands  in  response  to  the  recent 
direction  to  review  food  service  contracts,  and  take  actions,  as 
appropriate,  to  ensure  that  cost  savings  measures  are  implemented. 


Agency  Comments 
and  Our  Evaluation 


In  written  comments  on  a  draft  of  this  report,  DOD  stated  that  it  concurred 
with  all  of  our  recommendations  and  that  the  Air  Force  is  already  taking 
steps  to  implement  them.  For  example,  DOD  has  taken  action  to 
implement  our  recommendation  that  the  Secretary  of  the  Air  Force 
monitor  the  actions  taken  by  the  Air  Force  Major  Commands  to  review 
food  service  contracts,  and  take  actions,  as  appropriate,  to  ensure  that 
cost  savings  measures  are  implemented.  In  its  response,  DOD  stated  that 
each  Air  Force  Major  Command  has  tasked  its  bases  to  conduct  a  100 
percent  review  of  existing  food  service  contracts  to  determine  if  their 
current  contracts  require  modification  to  meet  current  mission 
needs.  DOD  further  noted  that  it  intends  to  share  the  results  of  the  Air 
Force’s  review  of  its  food  service  labor  costs  to  achieve  cost  savings  with 
the  other  services.  DOD’s  comments  are  printed  in  their  entirety  in 
appendix  II. 

DOD  also  commented  on  our  discussion  of  the  response  rate  for  the  Air 
Force’s  recent  customer  satisfaction  survey.  In  our  report,  we  stated  that 
the  low  response  rate  (17  percent  of  active  duty  airmen)  for  the  Air 
Force’s  annual  customer  satisfaction  survey  could  bias  the  data  it  uses  to 
measure  the  results  of  the  Food  Transformation  Initiative.  DOD  stated 
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that,  according  to  the  contractor  who  conducted  the  survey,  the  response 
rate  achieved  in  the  Air  Force’s  customer  satisfaction  survey  met  and,  in 
most  cases,  exceeded  the  response  rate  that  is  usually  achieved  for 
these  types  of  surveys.  However,  we  continue  to  believe  that  this 
response  rate  decreases  the  likelihood  that  the  survey  results  are 
representative  of  the  views  and  characteristics  of  the  population. 
According  to  the  Office  of  Management  and  Budget’s  standards  for 
statistical  surveys,  a  nonresponse  analysis  is  recommended  for  surveys 
with  response  rates  lower  than  80  percent  to  determine  whether  the 
responses  are  representative  of  the  surveyed  population.  Conducting  a 
nonresponse  analysis  would  allow  the  Air  Force  to  more  confidently  rely 
on  the  survey  results  in  assessing  the  results  of  the  initiative. 


We  are  sending  copies  of  this  report  to  appropriate  congressional 
committees,  the  Secretary  of  Defense,  and  the  Secretary  of  the  Air  Force. 
The  report  is  also  available  at  no  charge  on  the  GAO  website  at 
http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  regarding  this  report,  please 
contact  me  at  (202)  512-3863  or  moranr@gao.gov.  Contact  points  for  our 
Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on 
the  last  page  of  this  report.  GAO  staff  who  made  key  contributions  to  this 
report  are  listed  in  appendix  III. 

£ 

Revae  Moran 

Director,  Defense  Capabilities  and  Management 
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List  of  Committees 


The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  McCain 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Daniel  Inouye 
Chairman 

The  Honorable  Thad  Cochran 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Howard  P.  McKeon 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  C.W.  Bill  Young 
Chairman 

The  Honorable  Norman  D.  Dicks 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  I:  Scope  and  Methodology 


To  assess  the  Food  Transformation  Initiative’s  objectives  and 
performance  measures,  we  reviewed  pertinent  documentation  and 
interviewed  Air  Force  headquarters  and  Air  Force  Services  Agency 
officials  about  the  initiative’s  background,  goals,  and  how  the  results  of 
the  initiative  would  be  determined.  We  also  reviewed  Department  of 
Defense  (DOD)  and  Air  Force  guidance,  as  well  as  criteria  from  our 
previous  work,  for  developing  metrics  and  plans  for  assessing  the  results 
of  programs  such  as  the  initiative.  We  compared  principles  we  derived 
from  this  guidance  and  criteria  with  the  Air  Force’s  metrics  and 
assessment  plans  for  the  initiative  and  where  there  were  differences  we 
discussed  the  reasons  why  with  Air  Force  officials. 

To  review  the  Air  Force’s  progress  in  meeting  its  objectives  for  the 
initiative,  we  collected  available  quantitative  data  related  to  each  of  the 
initiative’s  objectives  and  compared  the  data  before  and  after 
implementation  of  the  initiative.  For  example,  we  obtained  and  reviewed 
changes  in  hours  of  operation  of  the  main  dining  facilities,  the  menus 
being  used,  the  number  of  meals  served,  and  the  cost  of  food  service 
contracts.  Also,  to  discuss  with  installation  officials  changes  made  under 
the  initiative  and  progress  towards  the  initiative’s  objectives,  we  visited 
four  of  the  six  installations  where  the  initiative  was  being  piloted — 
Elmendorf  Air  Force  Base,  Alaska;  Fairchild  Air  Force  Base,  Washington; 
MacDill  Air  Force  Base,  Florida;  and  Travis  Air  Force  Base,  California. 

We  selected  the  bases  to  visit  to  include  various  geographic  locations  and 
other  factors.  In  addition,  we  reviewed  the  Air  Force’s  customer 
satisfaction  survey  results  on  changes  in  the  reported  level  of  airmen 
satisfaction  with  various  aspects  of  their  food  service  experience.  In  areas 
where  initiative  progress  to  date  differed  from  Air  Force  plans,  such  as  in 
the  timing  of  the  implementation  of  the  campus  dining  concept  and 
changes  in  labor  costs,  we  discussed  with  Air  Force  officials  the  reasons 
why  and  the  associated  impact  on  measuring  the  progress  of  the  initiative 
at  the  pilot  installations.  In  examining  the  meal  count  data,  we  looked  for 
anomalies  by  comparing  data  month  to  month.  We  discussed  anomalies 
with  knowledgeable  officials.  As  a  result,  the  Air  Force  went  through  a 
revalidation  process  to  verify  the  numbers  and  provided  us  with  corrected 
data.  Therefore,  we  believe  that  the  meal  count  data  we  assessed  are 
sufficiently  reliable  for  the  purposes  of  this  report.  To  assess  the  reliability 
of  the  customer  survey  data,  we  discussed  the  data  with  knowledgeable 
officials  and  examined  the  survey  response  rate.  The  survey  response 
rate  is  low  and  the  Air  Force  did  not  perform  a  nonresponse  analysis  to 
clarify  whether  those  who  did  not  respond  to  the  survey  may  provide 
substantively  different  answers  than  those  who  did  respond.  The  survey 
results  could  be  interpreted  more  confidently  if  a  nonresponse  analysis 
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was  done  to  establish  whether  or  not  it  is  likely  that  there  are  any 
systematic  biases  due  to  some  people  being  more  or  less  likely  to 
respond  to  the  survey.  However,  we  are  presenting  these  data  as 
descriptive  information  that  the  Air  Force  is  using  to  measure  its  progress 
in  meeting  these  objectives  and  have  noted  its  limitations. 

To  document  the  initiative’s  impact  on  the  food  service  employees  at  the 
pilot  bases,  we  considered  the  three  types  of  food  service  employees: 
appropriated  fund  civilian  and  military  food  service  employees  at  the  main 
dining  halls,  contract  workers  at  the  main  dining  halls,  and 
nonappropriated  fund  employees  at  nonappropriated  fund  food  and 
beverage  operations.  Specifically,  for  the  civilian  and  military  food  service 
employees,  we  interviewed  union  representatives  and  groups  of  food 
service  workers,  as  well  as  individual  workers,  who  were  available  to 
meet  with  us  during  our  visit  about  their  experiences  under  the  initiative, 
and  also  spoke  to  Air  Force  and  Aramark  officials  about  how  the  initiative 
affected  these  employees,  including  any  changes  in  employment 
numbers.  For  the  contract  employees  at  the  main  dining  facilities,  we 
obtained  and  compared  the  number  of  employees  prior  to  and  after  the 
implementation  of  the  initiative  and  also  spoke  with  Air  Force,  Aramark, 
and  Ability  One  officials  about  changes  under  the  initiative.  Aramark 
officials  stated  that  the  numbers  represent  the  number  of  employees  the 
day  Aramark  took  over  the  contract  and  accurately  reflect  the 
terminations  and  hires  since  October  1,  2010.  However,  the  contract 
employee  numbers  cannot  be  validated  since  the  names  of  the 
individuals  are  not  available.  These  are  small  numbers  of  employees,  and 
Aramark  was  able  to  count  the  number  of  Ability  One  workers  at  the 
dining  halls  on  the  first  day  of  the  contract.  Because  these  are  small 
numbers  of  employees,  we  believe  there  is  less  possibility  for  error. 
Therefore,  we  believe  the  data  are  sufficiently  reliable  for  the  purpose  of 
this  report.  For  the  employees  at  the  nonappropriated  fund  food  and 
beverage  operations,  we  obtained  and  compared  the  number  of 
employees  prior  to  and  after  the  implementation  of  the  initiative  and 
documented  the  employment  options  taken  by  these  employees  after  the 
implementation  of  the  initiative.  We  also  interviewed  some  former 
nonappropriated  fund  food  and  beverage  operation  employees  who  were 
available  to  speak  with  us  during  our  visit  to  discuss  their  experiences 
under  the  initiative  and  discussed  with  Air  Force  headquarters  and 
installation  officials  employee  severance  payments  at  the  bases  we 
visited. 

To  assess  the  Air  Force’s  consideration  of  alternative  approaches  for 
transforming  its  food  service  program,  we  reviewed  the  Air  Force 
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business  case  analysis  that  was  used  to  evaluate  alternatives  for  meeting 
the  Air  Force’s  food  service  improvement  objectives.  Specifically,  we 
discussed  the  business  case  analysis  process  with  Air  Force  officials  and 
compared  the  methodology  used  in  the  analysis  with  Air  Force  guidance 
for  conducting  such  analyses  and  with  guidance  contained  in  the  GAO 
Cost  Estimating  and  Assessment  Guided  In  areas  where  the  business 
case  analysis  did  not  follow  guidance,  we  discussed  the  reasons  why  with 
Air  Force  officials  and  also  asked  about  options  for  meeting  the  Air 
Force’s  food  service  improvement  objectives  that  were  not  included  in  the 
analysis.  Further,  we  talked  with  Air  Force  officials  about  opportunities  for 
reducing  food  service  costs  outside  of  the  Food  Transformation  Initiative. 

We  provided  a  draft  of  this  report  to  DOD  officials  for  their  review  and 
incorporated  their  comments  where  appropriate.  We  also  provided  the 
pertinent  factual  information  contained  in  this  draft  to  the  contractor, 
Aramark,  for  review  and  incorporated  Aramark’s  comments  as 
appropriate. 

We  conducted  this  performance  audit  from  September  2010  to  July  201 1 , 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


^AO,  GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
Managing  Capital  Program  Costs,  GAO-09-3SP  (Washington,  D.C.:  March  2009). 
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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

4000  DEFENSE  PENTAGON 
WASHINGTON,  D  C.  20301-4000 


JUL  8  2011 

Ms.  Revae  Moran 

Acting  Director,  Delense  Capabilities  and  Management 
Unites  States  Government  Accountability  Office 
Washington  DC  20548 

Dear  Ms.  Moran: 

Thank  you  for  the  opportunity  to  comment  on  GAO  Draft  Report,  GAO-1 1-676, 
"DEFENSE  MANAGEMENT:  Actions  Needed  to  Improve  Management  of  Air  Force's  Food 
1  ranslormation  Initiative,'  dated  June  9,  201 1  (GAO  Code  351536).  The  Department  concurs 
with  the  four  recommendations  and  appreciates  the  comprehensive  review  the  GAO  conducted 
on  this  important  initiative.  The  enclosure  to  this  letter  outlines  the  actions  Air  Force  is  taking  in 
response  to  the  recommendations. 

Ol  particular  note  is  GAO's  recommendation  to  use  lessons  learned  from  the  Food 
Transformation  Initiative  to  review  and  renegotiate  current  food  service  contracts  to  adjust  labor 
costs  as  necessary  at  all  Air  Force  dining  facilities,  potentially  achieving  immediate  savings  prior 
to  expanding  the  initiative.  Wc  intend  to  share  this  lesson  learned  with  the  other  Services. 

With  regard  to  the  annual  customer  satisfaction  survey  that  Air  Force  will  use  to  measure 
progress  for  several  Food  Transformation  objectives,  it  is  important  to  clarify  the  survey 
methodology  and  reliability.  The  CFI  Group,  a  management  consulting  firm  conducts  the  survey 
and  specializes  in  the  application  of  the  American  Customer  Satisfaction  Index  (ACSI) 
methodology  to  individual  Air  Force  program  objectives.  The  ACSI,  established  in  1994.  is  a 
uniform,  cross-industry  measure  of  satisfaction  with  goods  and  services  in  a  number  of  national 
industry  sectors  from  transportation  and  health  care  to  hotels  and  food  sendee. 

CFf  group  uses  the  ACSI  methodology  to  identify  the  causes  ol"  satisfaction,  and  relates 
satisfaction  to  business  performance  measures.  Using  CFI  Group  results.  Air  Force  can  identify 
and  improve  those  factors  that  will  improve  satisfaction  and  other  measures  of  business 
performance.  ACSI  ratings  are  presented  on  a  0-100  scale.  Within  the  context  of  the  survey 
results,  scores  in  the  60s  are  characterized  as  “fine  but  could  use  work,"  the  70s  as  “good  job  but 
keep  working  on  if"  and  the  80s  as  “excellent  -  keep  it  up."  At  the  program  area  level,  CS1 
scores  in  the  mid-70s  are  expected;  it  is  unlikely  that  any  program  area  will  achieve  a  score 
greater  than  85.  An  increase  of  tw'o  points  or  more  is  considered  significant  for  increased 
customer  satisfaction  (and  business  performance).  Therefore,  the  comparison  on  page  1 5 
between  customer  satisfaction  with  several  criteria  for  the  pilot  bases  and  non  pilot  bases  show’s 
significant  improvement  in  customer  satisfaction  where  Food  Transformation  has  been 
implemented  and  Air  Force's  stated  goals  for  future  CFI  scores  are  commendable. 
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The  GAO  expresses  concern  about  a  low  (17  percent)  response  rate  among  active  duty 
personnel  for  the  recent  surveys.  In  consultation  with  CFI  group  personnel,  we  have 
benchmarked  the  achieved  response  rate  to  several  surveys  conducted  by  CFI  of  a  similar 
methodology  (i.e.,  “outbound”  email  invitation  with  some  pre-notification,  reminder/follow  up 
emails  sent,  no  incentives  offered  for  completion).  For  example,  a  survey  of  institutional 
investors  of  financial  services  had  a  24  percent  response  rate;  account  holders  at  credit  unions  a 
1  -8  percent  response  rate;  businesses  and  organizations  within  government  agencies  a  5-20 
percent  response  rate;  and  subscribers  to  satellite  television  a  3-5  percent  response  rate. 

The  response  rates  achieved  in  the  Air  Force  2010  Caring  for  People  survey  meet  and  in 
most  cases  far  exceed  the  rates  CFI  Group  is  accustomed  to  seeing  in  its  other  client  work.  For 
the  sample  sizes  used,  the  confidence  intervals  around  the  scores  are  within  2-3  points  on  the  0  to 
100  point  reporting  scale  at  the  90  percent  level  of  confidence  (some  programs  areas  will  be 
higher),  which  is  sufficient  to  identify  substantive  differences  in  performance  over  time  and/or 
across  various  segmentations  of  the  data. 

Thank  you  again  for  the  excellent  review.  The  professional  and  experienced  conduct  of 
you  and  your  staff  was  exceptional  and  contributed  to  a  very  productive  study. 


^—-"'"’"Robert  L.  Gordon  III 

Deputy  Assistant  Secretary  of  Defense 
(Military  Community  and  Family  Policy) 


Enclosure: 
As  stated 
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GAO  DRAFT  REPORT  DATED  JUNE  9,  2011 
GAO-1 1-676  (GAO  CODE  351536) 

“DEFENSE  MANAGEMENT:  ACTIONS  NEEDED  TO  IMPROVE 
MANAGEMENT  OF  AIR  FORCE'S  FOOD  TRANSFORMATION 
INITIATIVE” 

DoD  COMMENTS 

TO  THE  GAO  RECOMMENDATIONS 


RECOMMENDATION  1:  Before  implementing  the  Food  Transformation 
Initiative  at  other  Air  Force  bases,  the  GAO  recommends  that  the  Secretary  of  the 
Air  Force  direct  the  Air  Force  Services  Agency  to  develop  an  evaluation  plan  to 
accurately  and  reliably  assess  the  results  of  the  pilot  phase  of  the  Food 
Transformation  Initiative.  This  evaluation  plan  should  include,  at  a  minimum, 
well-defined,  clear,  and  measurable  metrics  to  use  in  determining  the  results  of  the 
initiative  and  time  frames  for  collecting  data  and  reporting  results. 

DoD  RESPONSE:  Concur.  Air  Force  Services  will  complete  the  evaluation  plan, 
with  well  defined  measureable  metrics,  in  August  2011.  There  are  currently 
measureable  metrics  for  seven  of  the  eight  Air  Force  objectives,  and  work 
continues  on  the  development  of  measureable  data  for  the  eighth  objective, 
“providing  a  sense  of  community.”  Final  efforts  are  being  made  on  the  Balanced 
Scorecard  and  evaluation  plan,  and  the  completed  plan  will  be  in  effect  well  before 
implementation  of  Food  Transformation  at  the  next  portfolio  of  bases. 

RECOMMENDATION  2:  Before  implementing  the  Food  Transformation 
Initiative  at  other  Air  Force  bases,  the  GAO  recommends  that  the  Secretary  of  the 
Air  Force  direct  the  Air  Force  Services  Agency  to  evaluate  the  results  of  the 
implementation  of  the  campus  dining  concept  at  the  six  pilot  locations  to 
determine  if  adjustments  are  needed  before  expanding  the  initiative  to  additional 
bases. 

DoD  RESPONSE:  Concur.  The  implementation  of  Campus  Dining  began  June 
6,  201 1  at  Patrick  AFB.  Implementation  of  Campus  Dining  at  all  six  pilot  bases  is 
scheduled  for  completion  by  the  end  of  July  2011.  As  each  implementation 
occurs,  Campus  Dining  will  be  subject  to  initial  and  ongoing  analysis,  with 
adjustments  made  as  issues  are  identified.  The  Air  Force  delayed  Campus  Dining 
until  an  effective  method  for  capturing  Essential  Station  Messing  (ESM)  usage  at 
nonappropriated  fund  (NAF)  food  service  operations  could  be  implemented.  The 
requirement  to  discontinue  use  of  the  member’s  social  security  account  number  as 
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a  means  to  identify  enlisted  members  on  ESM  required  the  identification  and 
development  of  an  alternative  method  of  identifying  ESM  members  and  the  meals 
they  consume.  Use  of  the  alternate  identifier  on  a  military  member’s  identification 
card  was  pursued,  and  the  appropriate  software  product  developed.  Air  Force  is 
confident  they  can  now  provide  a  Campus  Dining  benefit  to  enlisted  members  on 
ESM  with  the  level  of  accountability  required. 

RECOMMENDATION  3:  Before  implementing  the  Food  Transformation 
Initiative  at  other  Air  Force  bases,  the  GAO  recommends  that  the  Secretary  of  the 
Air  Force  direct  the  Air  Force  Services  Agency  to  reevaluate  the  number  of 
military  personnel  hours  the  Air  Force  can  provide  the  contractor  for  food  service, 
make  adjustments  where  possible  in  projected  labor  hours,  and  take  into  account 
the  increased  labor  costs  in  revising  the  cost  savings  estimates  for  the  future. 


AF  RESPONSE:  Concur.  The  Air  Force  Services  Agency  (AFSVA)  has  tracked 
the  availability  of  military  food  service  manpower  since  the  initial  implementation 
of  Food  Transfonnation,  and  this  process  continues.  The  pilot  bases  are  currently 
conducting  a  100%  revalidation  of  available  military  manpower  for  use  in  a  final 
4th  quarter  FY1 1  cost  projection,  and  for  determining  the  cost  estimate  for  their 
food  service  contract  for  FY12.  Applying  the  lessons  learned  from  the  initial 
implementation,  additional  planning  factors  for  use  in  determining  the  amount  of 
available  military  food  service  manpower  were  sent  to  each  base  in  the  next 
portfolio  and  their  parent  Major  Command.  Once  the  final  manpower  estimates 
are  provided  to  AFSVA,  the  Food  Transformation  team  will  contact  each  base  to 
insure  all  planning  factors  have  been  properly  considered  prior  to  releasing  any 
due  diligence  data  for  contract  bidder  use  on  the  next  portfolio. 

RECOMMENDATION  4:  The  GAO  recommends  that  the  Secretary  of  the  Air 
Force  monitor  the  actions  taken  by  the  Air  Force  Major  Commands  in  response  to 
the  recent  direction  to  review  food  service  contracts,  and  take  actions,  as 
appropriate,  to  ensure  that  cost  savings  measures  are  implemented. 

DoD  RESPONSE:  Concur.  On  March  28,  201 1,  the  Commander,  AFSVA 
requested  each  Major  Command  task  their  bases  to  conduct  a  100%  review  of 
existing  food  service,  to  determine  if  their  current  contract  workload  estimates 
meet  current  mission  needs  or  if  the  contract  requires  modification.  Results  of  this 
review  are  currently  being  analyzed. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday  afternoon, 
GAO  posts  on  its  Web  site  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  Web  site, 

http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Ralph  Dawn,  Managing  Director,  dawnr@gao.gov,  (202)  512-4400 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125 
Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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